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Use of the Square-Root Method to Identify Factors in the Job 
Performance of Aircraft Mechanics' 


CHARLES WRIGLEY, CHARLES N. CHERRY,’ 
MARILYN С. LEE, AND Louis L. McQurrry' 
University of Illinois 


I. INTRODUCTION 


uis report has a threefold purpose: 
To to exemplify application of a 
modified version® of square-root factor 
analysis to a large correlation matrix, 
(b) to describe the factorial structure of 


1 This research was supported in part by the 
United States Air Force under Contract No. 
33-(038)-25726 monitored by the Air Force Per- 
sonnel and Training Research Center. Permis- 
sion is granted for reproduction, translation, 
publication, use, and disposal in whole and in 
part by or for the United States Government. 

* Now at the University of California. 

“Now at State Farm Insurance Companies, 
Bloomington, Illinois. 

“This study was a cooperative project whose 
completion depended upon the assistance of 
many persons. Kern W. Dickman and Mrs. Carol 
R, Tucker carried out the statistical analyses. 
Mr. Dickman also prepared a computer program 
for rank-ordering sets of correlations and factor 
loadings which greatly reduced the amount of 
hand computation required, Ramona J. Russell 
and William L. Huffman supervised the com- 
and Robert D. Boys and Mar- 
vin I, Knopp assembled and organized the sta- 
tistical results. The authors wish to express their 
sincere appreciation to the persons named for 
their assistance and cooperation. 

?'The modified square-root method of factor 
analysis was developed by Wrigley and McQuitty 
(9) in an attempt to meet certain criticisms cus- 
tomarily advanced against the square-root 
method, with a view to making, factor analysis 
more readily applicable to large correlation 
matrices. The nature and purposes of the 
changes are briefly described in Section IIT of 
this report. z 


puter operations, 


job performance ratings obtained from 
supervisors of aircraft mechanics and 
(c) to compare the results of this study 
with those of two earlier and similar in- 
vestigations (6, 7) in order to determine 
the most appropriate source of items for 
the rating of job performance in the 
field of aircraft maintenance. 

Taking up the third purpose first, it 
should be explained that in the first 
study of the series (7), the descriptive 
items to rate job performance were ob- 
tained from interviews with supervisors 
of aircraft mechanics. In the second (6), 
items were selected from a wide variety 
of factor analyses of personality traits 
appearing in the psychological litera- 
ture. In the third, reported here, items 
were drawn from existing rating scales 
and questionnaires designed to measure 
various aspects of job performance. 

Each phase of the research led to 
preparation of a different inventory in- 
tended for use as a rating device. The 
development and analysis of each inven- 
tory was conducted in accordance with 
the same general plan. An inventory of 
200 items was obtained by assembling 


“The subjects of this study were Air Force 
Aircraft and Engine Mechanics. To conserve 
space, they will throughout be referred to as 
"aircraft mechanics." 
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a pool of items and revising a sample of 
them. This was administered to a large 
sample of supervisors of aircraft me- 
chanics. (N's for the three studies ranged 
[rom 383 to 464.) Approximately one 
quarter of the subjects in each sample 
were asked to describe a “best,” one half 
to describe an "average," and one quar- 
ter to describe a "poorest" mechanic, by 
responding "true" or "false" to the in- 
ventory items. Each inventory was then 
factorized in order to permit the develop- 
ment of a single, more comprehensive 
inventory in which overlapping of items 
in terms of factorial content would be 
minimal. 

It should be noted here that while in 
principle factor analysis is generally con- 
sidered to be the most satisfactory pro- 
cedure currently available for reducing 
the number of items or variables with 
least loss of information, it suffers from 
a serious limitation, namely, computa- 
tions are simply too laborious for appli- 
cation to any very large set of variables. 
In the present series of studies, where 
so many items had be considered, a 
simpler procedure was needed than cal- 
culation and rotation of a centroid or 
a multiple-group factor analysis. The 
modified square-root method was accord- 
ingly used. A virtue of this procedure is 
that it can be applied without incom- 
mensurate labor to as many as 200 or 
250 variables, provided reasonably good 
computational facilities are available. 
Because of its potential usefulness in 
other studies where the number of vari- 
ables to be analyzed is very large, this 
report is devoted in part to a brief de- 
scription of the method. 

The organization of this monograph is 
as follows: The second section describes 
the preparation and administration of 
the job performance inventory, the third 


describes the square-root method ol Гас 
tor analysis, and the fourth and fifth 
sections present the results of the statis- 
tical analyses of the data. In the sixth 
section, results of the present study are 
compared with those of the two which 
preceded it in the series. The study is 
summarized in the seventh and final 
section. 


II. PREPARATION AND ADMINISTRATION 
or THE “SuRVEY INVENTORY” 


The psychological literature contains 
many rating scales апа questionnaires 
designed to measure various aspects of 
vocational proficiency. Few of these 
studies refer directly to the selection 
of mechanics. However, many deal with 
jobs which appear to be somewhat simi- 
lar, and include items referring to traits 
of possible importance to success in air- 
craft maintenance work. 

It was therefore hypothesized that 
items drawn from these studies would be 
useful for describing job performance of 
aircraft mechanics. The content of these 
scales is broad in scope, and each item 
reflects the accumulated experience ol 
one or more experts in personnel psy- 
chology. Appropriate sampling of this 
material should thus contribute to the 
attempt to isolate characteristics related 
to efficient aircraft maintenance. 

Collection and sampling of the item 
pool. In order to locate pertinent studies, 
the Psychological Index and Psychologi- 
cal Abstracts from 1900 to 1952 Were con- 
sulted for articles containing relevant 
rating scales and questionnaires, The 
search yielded 64 studies containing de- 
scriptive phrases which appeared to be 
related to the on-the-job behavior of air- 
craft mechanics. From these sources. 
some 3,365 items were chosen, 

These items were then classified into 
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42 content categories according to a 
method developed by Staugas and Mc- 
Quitty (8), and a stratified sample was 
drawn from the pool in such a way as 
to minimize overlap with the two pre- 
ceding inventories. In selecting items for 
the sample, preference was given to those 
which (a) were expressed in simple 
language and (b) appeared to represent 
most clearly the essence of the category 
into which they had been classified. To 
allow for rejection of items which might 
prove unsatisfactory upon preliminary 
administration, 221 rather than 200 
items were initially selected from the 
pool. 

Modification of items and assembly 
of the inventory. In assembling the in- 
ventory, the aim was to retain each 
item in the exact form in which it was 
written whenever possible, and if any 
alterations in wording had to be made, 
to insure that the original meaning of 
the item remained unchanged. The 
following types of editorial revisions 
were made: 


1. Items calling for self-judgments меге 
changed to a form requiring judgments of 
others. This was done because the inventory 
was intended for use by supervisors for the rat- 
ing of line workers. 

2. Items referring to supervisory personnel 
were modified for application to line mechanics. 

g. Items relating to civilian jobs were re- 
worded to apply more specifically to Air Force 
maintenance work. 

4. Items designed for females were changed 
to apply to males, 7 

5. Items іп the form of a question were 
turned into declarative statements. 

6. Some items were shortened to improve 
their clarity. 

7. The grammatical form was rendered con- 
sistent from item to item. 

8. Words judged by the investigators to be 
difficult were replaced by words believed to be 
more commensurate with the educational attain- 
ments of the average Air Force mechanic. 

9.In a “double-barrelled” item, the more 
general phrase of the two was retained, and 


э 


the qualifying, subordinate, or more specific 
phrase eliminated. 

10. All items were cast into the present or 
the conditional tense for consistency and ease 
of interpretation. 


Upon completion of editing, the 
phrases were assembled into a prelimi- 
nary inventory and administered indi- 
vidually to 28 aircraft mechanics, all of 
whom had had at least six months of 
experience on the line. Half of these sub- 
jects were asked to rate the "best" and 
half were requested to rate the "poorest" 
mechanic with whom they had worked, 
by checking “true” or “false” for each 
phrase in the inventory. (The two-point 
scale of “true” and “false” was decided 
upon because of our intention to use 
phi coefficients in the factor analysis.) 
The entire group of subjects was en- 
couraged to make comments on the in- 
dividual items. 

After the responses had been tallied, 
chi square was calculated for each item 
in order to locate those which failed to 
discriminate significantly between the 
descriptions of “best” and “poorest” me- 
chanics. The items were then reviewed 
by three judges. Those which met all of 
the following conditions were retained 
without modification: 

1. The chi-square value was significant at 
the .05 level. 

2. No comment concerning either wording 
or understanding of the item was offered during 
individual administration. 

3. Neither answer alternative to the item 


was endorsed by fewer than 10 per cent of the 
subjects. 


These requirements were met by 167 
of the items. The remaining items were 
considered one at a time with respect to 
the above three criteria. Items with less 
than a 10 per cent response for either 
answer alternative were rewritten to en- 
large this percentage whenever this ap- 
peared possible; otherwise they were re- 
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jected. Items which mechanics reported 
as requiring information that they did 
not have were rejected, and those difficult 
to understand were amended. This pro- 
cedure yielded a final inventory of 200 
items, named by us the Survey Inventory. 
This is the instrument used in all sub- 
sequent phases of this study. It is re- 
produced in the Appendix. 

Administration of the inventory. 'Yo 
obtain data for the factor analysis, the 
inventory was administered to 464 super- 
visors of aircraft mechanics from Cha- 
nute, Sheppard, and Connally Air Force 
Bases. In this administration, 148 super- 
visors were asked to describe the “best,” 
179 were asked to describe an “average,” 
and 137 were asked to describe the 
“poorest” mechanic with whom they had 
worked during the previous two years. 
The inventory was given in small group 
sessions (10 to зо men) of about 30 
minutes in length. This allowed ample 
time for all subjects to complete the 
items. 

For each subject, the following occu- 
pational experience was required: (а) 
completion of the basic training course 
for aircraft mechanics; and (b) at least 
Six months' experience as a supervisor 
of aircraft mechanics. 

The data obtained in this phase of the 
study were then subjected to a modified 
square-root factor analysis. These results 
are presented in Section IV. 


III. THe SQUARE-ROOT METHOD or 
Facror ANALYSIS 

The standard methods of factor analy- 
sis are generally agreed to be too labori- 
ous to be readily applicable to a correla- 
tion matrix much larger than 100 vari- 
ables. Yet there are occasions when it 
would be desirable to analyze correlation 
matrices of a size considerably larger 


than this. The square-root method о! 
factor analysis (sometimes known as the 
diagonal method) is the simplest factor- 
analytic procedure, but has not WOR 
widespread favor because of various 
criticisms advanced against it. The meth- 
od has accordingly been modified by 
Wrigley and McQuitty (9) in 27 attempt 
to meet these criticisms; in the course ol 
doing so, the computations have been 
still further simplified. . 
Our restricted objective in making 
these modifications should be made quite 
explicit. The intention was not to pro 
vide a method which is in any sense tO 
be regarded as a competitor of standard 
methods; the purpose was merely to pie" 
vide some way of analyzing correlation 
matrices too large for the regular pro- 
cedures. Hence it is not incumbent upon 
us to demonstrate that our solution is as 
satisfactory as, Say, а simple structure 
solution would be, even were it possible 
to do so. It suffices if we can demonstrate 
that square-root factors are generally in- 
terpretable and psychologically meaning- 
ful, and add in some measure to our 
understanding of the structural inter- 
relations of the variables analyzed. 
The square-root method. First let us 
briefly summarize the standard form of 
the square-root method as well as the 
criticisms which have been made of it. 
The first step in the procedure is to 
calculate loadings for a first factor in 
such a way as to reduce all residual cor- 
relations with a selected pivot variable 
to zero. This is equivalent to placing a 
reference axis collinear with the test 
vector of the pivot variable. Residual 
correlations are then calculated in the 
usual way. This implies projection of 
the test vectors onto a plane orthogonal 
to the reference axis. In the next step, 
loadings are calculated for a second fac- 
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tor so that residual correlations for a 
second pivot variable are reduced to zero. 
The second axis will be collinear to the 
residual test vector of the second pivot 
variable, and will be at right angles to 
the first. The second set of residual cor- 
relations is calculated, and so the method 
proceeds. Since successive reference axes 
are at right angles, the factors are mutu- 
ally orthogonal, so that the factor meas- 
urements for the persons in the sample, 
if calculated, would be uncorrelated. 

The computational formula for 
square-root loadings is simple. The factor 
loading f;, for the ith variable on the 
first factor is: 


VT qq 


where r;, is the correlation of the ith 
variable with the pivot variable q and 
T,, is the diagonal entry for the pivot 
variable. 

More generally, the loading for the 
jth factor, with pivot variable s, is: 


paf 
EZ 
where rj,U-D, r- are the residual 
correlations for the (j-1)th factor. 
Criticisms of the method. There have 


been two main criticisms of the square- 
root method: 


ij 


1. The procedure depends to a greater extent 
than most other factor-analytic methods upon 
accurate estimation of the communalities, since 
diagonal entries play such an important role 
in calculating the factor loadings. If some com- 
munalities are underestimated, then some diago- 
nal entries may become negative during the 
calculations. This means that one must take 
the square root of a negative number, which 
results in factor loadings which are imaginary. 

2. The order of selection of pivot variables 
is unspecified, so that p factorial solutions are 
possible (where p — the number of variables). 
No rule is provided for selecting any one order 
of pivot selection in preference to any other. 


The modified procedure. Our modifi- 
cations were intended to provide (a) a 
logical basis for specifying order of selec- 
tion of pivot variables, (b) a method not 
dependent in any way upon the arbitrary 
judgment of the investigator, and (c) a 
computational procedure rapid enough 
to permit the handling of large correla- 
tion matrices. The following three modi- 
fications have accordingly been intro- 
duced: 


1. Unities are inserted in the leading diagonal 
in place of communalities. This insures that so 
long as product-moment correlations are ana- 
lyzed, no diagonal entry can become negative 
during the calculation: (Phi coefficients and 
point biserials will be remembered to be special 
forms of product-moment correlation.) 

It is true that if unities are used, the factors 
obtained include specific as well as common 
variance. There are some who have doubted 
Whether any factors derived by the use of 
unities rather than communalities can be psycho- 
logically meaningful. Our experience has been 
that they are interpretable. It will be noted 
that the specific variance in any factor is re- 
stricted to the loading for the pivot variable. 
The other loadings represent only common 
variance. (It can be shown that the sum of 
squares of loadings for any nonpivot variable is 
equal to the squared multiple correlation with 
the set of pivot variables, і.е., is a measure of 
common variance only.) An incidental result 
of using unities is that factor measurements, 
if required, can be calculated exactly instead 
of having to be estimated, as is necessary in 
any communalities solution (5, ch. 12), 

2. The variable with the largest sum of off- 
diagonal correlations (and subsequent residual 
correlations) is selected as the pivot variable. 
This provides an objective criterion for pivoting 
order, and insures every factor will have some 
large loadings and that these will be predomi- 
nantly positive. 

3. A formula, based upon an algebraic iden- 
lity, was developed for calculating the sum of 
residual correlations for each variable without 
having to compute each individual residual. 
In the extraction of each factor, therefore, there 
must be calculated only the p column-sums, 
followed by p residuals for the indicated pivot 


*Ford, L. R., Jr. A proof that the matrices of 
certain indices of association are Gramian. Un- 
published manuscript, 1952. 
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variable, instead of the usual p (p + 1)/2 resid- 
ual correlations. 


Nonrotation of square-root factors. 
Rotation is generally essential for a prin- 
cipal axes or a centroid solution because 
the axes tend to pass between, rather 
than through, the test clusters. In the 
square-root method, however, the refer- 
ence axes are placed collinear with the 
pivot variables. Hence if each pivot lies 
in a fairly central position in a cluster, 
the reference axis will pass through the 
cluster. This generally turns out to be 
the case, since in the present method the 
pivot variables are selected on grounds 
of their high column-sums. For this 
reason, no rotation of the factor loadings 
is considered necessary. 

Parenthetically, it should be noted that 
when the data are dichotomous, our 
preference is for calculating phi coeffi- 
cients rather than tetrachoric correla- 
tions. The correlation matrix is then nec- 
essarily Gramian, so that no diagonal en- 
try can become negative in the course of 
the analysis (unless as а result of round- 
ing errors). 

The amount of variance accounted for 
by any factor in an orthogonal solution 
is, as is well known, a function of the 
sum of squares of the loadings. In the 
principal axes method, the sum of 
squares of each successive factor is neces- 
sarily equal to or smaller than those for 
preceding factors, and this relation is 
generally true for the centroid method. 
In our method of analysis, factors are 
not necessarily extracted in order of con- 
tribution of variance. However, because 
of pivot selection depending on column. 
sums, there is a marked tendency for 
larger factors to be extracted first and 
smaller factors later. (See Table 1 for 
evidence of this progression.) Our com- 
putational procedure has been to extract 


more factors than needed, this being 
practicable because of the speed of the 
method, and then to order them accord- 
ing to sums of squares of loadings, re- 
taining and reporting only the larger 
ones. 

The generally smaller size of square- 
root loadings. The square-root loadings 
obtained by the above procedure tend, 
in general, to be lower than would be 
obtained by a standard solution, such as 
a simple structure rotation of a centroid 
analysis of tetrachoric correlations. There 
are three main reasons for this: 

1. Phi coefficients are generally smaller ‘than 
tetrachoric correlations. : 

2. Even for the same correlation matrix, 
square-root loadings tend to be lower than 
centroid loadings. The sum of squares of load- 
ings for any nonpivot variable in a square-root 


solution can be shown to be equal to the 
squared multiple correlation (SMC) of that 
veriable with the k pivot variables. But the 
SMC of any variable with the k pivot variables 
only is necessarily less than or equal to the 
SMC of the variable with the full set of (p-1) 
other variables in the analysis. Guttman (4, pp- 

2-93) has shown this latter to constitute a 
lower bound for the communality. Hence the 
sum of squares of loadings for any nonpivot 
variable is necessarily less than or equal to the 
communality. In practice it is generally appre- 
ciably less. 

3. A general factor is more likely in a square 
root than in a simple structure solution, for 
the reason that the variable with largest column- 
sum is usually some rather generalized measure. 
Because more variance is accounted for by the 
first factor, less therefore remains to define 
later factors. 


Any new psychometric procedure has 
to establish two points: (а) its computa- 
tional feasibility, and (b) its psychologi- 
cal usefulness, The first requirement is 
clearly met in the present instance. The 
modified square-root method is consider- 
ably more rapid than any other factor- 
analytic procedure currently available. 
primarily because there is no longer any 
problem either of reflection or of rota- 
tion. In addition, the number of residual 
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correlations to be calculated in extract- 
ing each factor is much reduced. The 
computational gain is particularly evi- 
dent on an electronic computer. On 
Illiac (the University of Illinois com- 
puter), approximately six factors can be 
extracted from a correlation matrix of 
order goo in an hour. This time refers, 
of course, only to machine operation, and 
much further time may be needed for 
preparing tapes, printing results, etc. 
Even so, the factor analysis of a large 
correlation matrix becomes quite feas- 
ible. 

The main problem is therefore 
whether the classification of variables 
provided by the square-root method is 
psychologically meaningful, although 
unities have been inserted in the diag- 
onal, so that results may properly be 
used to reduce the number of variables 
to more manageable proportions. The 
data in the next section provide some of 
the evidence in support of our assertion 
that the factors provided by the modified 
square-root method are both interpret- 
able and informative. 


IV. RESULTS OF THE FACTOR ANALYSIS 


In obtaining the phi coefficients for 
the factor analysis, all subjects (whether 
classified as "best," "average," or “роог- 
est" mechanics) were included in a single 
sample. The square-root factor analysis 
of the phi coefficients was continued 
until 40 factors had been extracted. This 
extensive factoring was carried out to 
determine (a) the relative size of the suc- 
cessive factors, and (b) the number of 
square-root factors which appear to be 
psychologically meaningful, in a correla- 
tion matrix as large as this. 

Table 1 presents sums of squares of 
loadings for the до factors and indicates 
their order of extraction. These factors 


TABLE r 


SovanE-Roor FACTOR ANALYSIS: SUMS OF 
SQUARES OF Factor Loapincs 


7 Factor Sum of Factor Sum of 
„Factor im order squares Factor in order squares 
in order “of ГЇ: in order 

of size „Î. of factor of size ‚О! ex- of factor 
traction loadings traction loadings 
X r 53.6714 21 17 1.346r 
2 5 9.2086 22 16 1.3385 
3 2 7.7215 23 18 1.3240 
4 6 3.3006 24 27 1.2724 
5 8 2.9307 25 25 1.2705 
6 3 2.8628 26 28 1.2492 
Г 4 2.1757 27 21 1.1665 
8 ° 2.0308 28 23 1.1340 
9 12 1.8600 29 38 1.1257 
то 14 1.8280 зо 36 1.0190 
її 13 1.7899 31 26 .9507 
12 29 1.7800 2 31 +9473 
13 11 1.7171 33 32 +0434 
14 7 1.6961 34 33 +9228 
15 15 1.5993 35 35 +9169 
16 22 1.5618 36 зо .0121 
17 24 1.5545 37 34 +8230 
18 19 1.5001 38 40 -8139 
то 20 1.4520 зо зо ‚7894 
20 то 1.4250 4o 37 ‚1411 
Percentage 
Factors Sum of squares contribution 
of factor loadings to total 
variance 
т 53.6714 26.8% 
2-10 33.9286 17.0 
11-20 16.0767 8.0 
21-30 12,2483 б.т 
31-40 8.7606 m 
Remaining variance 75.3144 37.7 
Total (200 items) 200.0000 100.0% 


together account for 62.3 per cent of the 
total variance. The total variance is 
based upon all items, and is equal to the 
number of items, since each item has 
been taken to have unit variance. If all 
factors had been extracted, the total sum 
of squares of loadings would have 
equalled the total variance (i.e., 200). 
The 28 largest of the extracted factors 
appeared sufficiently meaningful to be 
named. However, owing to space limita- 
tions, only the first ten will be reported 
here.5 A second restriction involves the 


* А 19-page description of Factors 11 to 28 has 
been deposited with the American Documenta- 
tion Institute. Order Document No. 4839 from 
ADI Auxiliary Publications Project, Photodupli- 
cation Service, Library of Congress, Washington 
25, D.C., remitting in advance $1.75 for micro- 
film or $2.50 for 25 photocopies. Make checks 
payable to Chief, Photoduplication Service, Li- 
brary of Congress. 
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number of loadings presented for each 
factor. It is customary, in reporting 
factorial studies, to present all variables 
with loadings equal to or greater than a 
stated size—often .3o—and to exclude all 
with loadings less than that. In this 
study, however, there were 173 loadings 
above .go for the first factor, so that we 
have reported only the highest go, since 
these suffice to describe the factor. With 
respect to the remaining factors, the 
conventional cutoff point of .30 would 
be inappropriate because, as previously 
pointed out, the square-root loadings 
for factors subsequent to the first are 
lower than their simple structure coun- 
terparts. We have therefore uniformly 
used the ten highest loadings, even when 
lower than .$о, to determine whether the 
factor is meaningful. 

In the following discussion we shall 
use the term criterion correlation to de- 
signate the phi coefficient between the 
"true" or "false" responses to each item 
and the supervisors’ classifications of the 
mechanics as either “best” ог “poorest,” 
i.e., excluding “average” mechanics from 
the sample. The primary reason for ex- 
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cluding them in calculating these co- 
efficients was the availability of an elec- 
tronic computer program for dichot- 
omous data. Exclusion of the average 
group was preferred to combining it with 
one of the other classes because we were 
interested only in the comparative in- 
dices of validity. By computing the 
criterion correlations on the extreme 
cases only, the interitem comparisons 
were made as sharp as possible. 

It will no doubt be apparent that these 
criterion correlations аге probably 
heavily loaded by halo effects. When the 
supervisor selects а “poorest” mechanic 
for description, he is obviously likely to 
describe him as using good judgment in 
most situations, or as being respected by 
his co-workers. However, the probable 
existence of such effects does not make 
the criterion correlations valueless. By 
means of the criterion correlations, we 
can determine which attributes form part 
of the supervisors’ global impressions (or 
halo effects) and which do not. 

"The ten largest factors obtained by the 
square-root analysis will now be re- 
ported. 
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Factor 1. General Job Efficiency 


number Leading correlation Descriptive phrase 
86 1.000 -775 Does jobs in order of importance 
187 ‚704 -870 Fits well into the organization _ 
116 +732 .86s Goes over own work to see that it was done correctly 
161 .722 .816 Works well without supervision 
131 .720 .809 Sees what will be needed later and gets ready for it 
8 .603 -857 Seems to have a good future as a mechanic 
169 „бот .812 Works well without а lot of explaining 
123 . 688 -799 Оп the lookout for better ways of doing the job 
180 .688 .838 Не gives his best effort to whatever he is doing 
101 .688 .846 Sets a good example through his habits of work 
163 .686 .834 Handles additional and special responsibilities well 
92 ‚684 -834 Не should develop into one of the leading men in the crew 
145 ‚684 .808 Сап take over in an emergency 
179 . 683 .807 Is determined to make good 
200 . 68o .784 Is always a willing worker 
or . 671 .836 Very valuable in a new operation 
106 . 660 .791 Very good at going ahead alone 
152 .667 -773 Respected by all who know him 
144 .666 .745 Helps others with their work whenever possible 
до ‚664 .731 Selects proper Tech. Orders and makes correct use of them 
тїї ‚662 .797 He makes the best use of his ability 
34 „657 -769 Uses good judgment in most situations 
174 -657 .811 Knows how to organize to get right things done at right time 
114 — .686 —.806 Loses too much time in waste motion 
170 —.676 —.790 Never has things ready when needed 
121 —.670 —.795 Always ready with an excuse 
66 —.664 —.708 Seldom plans work ahead 
65 —.650 —.824 Takes far too many breaks 
88 —.650 —.797 Never notices things to be done 
95 — .656 —.848 Doesn't do his share of the work 


The criterion correlations are uni- 
formly large for highly loaded items on 
this first square-root factor. The factor 
might perhaps be interpreted as the sta- 
tistical representation of halo effects, and 
given the label of “General Success on 
the Job.” Our reason for not doing so is 
that we believe this title is too broad, 
and does not make clear the psychologi- 
cal nature of this general factor. The 
term “General Job Efficiency” was chosen 
instead, as better representing the es- 
sential import of the factor. The listed 
items stress the mechanic's ability to or- 
ganize his schedule so that his own work 
is coordinated with that of others and 
so that the more important jobs are 
done first. Such a mechanic can work by 
himself without detailed explanation or 


constant supervision. His working habits 
are good, and we are told that he gives 
his best effort to whatever he is doing, a 
characteristic suggesting that his effi- 
ciency is a function of high motivation as 
well as of sound organization, The factor 
is perhaps more revealing in terms of 
what is omitted than what is included. 
It should be noted that little is said ex- 
plicitly about the mechanic's manual 
dexterity or his knowledge of his job. 
and no reference is made to his social re- 
lations with the other mechanics, or to 
his emotional structuring. In this fac- 
tor, the stress seems to be not on what 
the mechanic himself is like, but rather 
on his capacity for working efficiently 
and making good use of the skills at his 
command. 
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Factor 2. Social Maladjustment 
Item Loadin Criterion Descriptive phrase 
number acing correlation puyexpure 
148 .769 —.736  Exaggerates too much 
112 .386 —.703 Uses a "line" to try to impress others 
146 .364 —.780 Wastes time walking around and talking to others 
81 .356 —.653 Rubs people the wrong way 
142 .351 —.673 Makes trouble over little things 
184 .349 —.694 Always wanting sympathy 
139 .324 —.483 Не butts in on others’ responsibilities 
130 . .321 —.752 Says one thing and does another 
173 .319 —.507 Tries to flatter crew chief 
200 —.318 .784 Is always a willing worker 


This factor stresses personality inade- 
quacies which result in unpopularity and 
generally poor interpersonal relations. 
The over-all flavor is one of immaturity, 


self-centeredness, and obtrusiveness. It 
should be noted that the traits described 
tend to be correlated with disinclination 


for work. 


Factor 3. Executive Ability 


—— 


M. Loading Le aW Descriptive phrase 
166 .827 .621 Discusses plans frequently with superior 
181 .316 .5о4 Ап extremely good organizer of personnel 
163 .310 .834 Handles additional and special responsibilities well 
40 .304 .635 Seeks and does additional tasks beyond those required 
174 .286 ‘811 Knows how to organize to get right things done at right time 
92 .284 .834 Не should develop into one of the leading men of the crew 
191 .280 .846 Sets a good example through his habits of work 
165 1275 .612 Quick to praise a man for a job well done 
122 — .282 —.568 Goes his own way regardless of others 
136 —.275 — 621 Dissatisfied with his present job 


Factor 3 defines the ability to organize 
activities and personnel and to spur co- 
operative achievement. The emphasis is 
not so much on the tendency to direct 


and dominate others as on the capacity 
for responsible supervision and effective 
administration. 


ш ЕЕЕ 
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Factor 4. Leadership 


E CUN Loading Sie Descriptive phrase 
63 .822 .641 He is apt to be master of a situation if it needs a leader 
21 .267 .5o5 Likes to take on responsibilities : 
189 .256 .523 He is good at organizing new activities 
134 =255 .659 Plans ahead for almost any emergency 
6o .253 -449 Не makes definite decisions and defends them with courage 
181 ‚251 -594 Ап extremely good organizer of personnel 
4 +243 .364 Likes to make decisions 
106 „212 .7от Very good at going ahead alone 
182 —.271 —.407 Prefers to be a follower rather than a leader 
67 —.208 —.795 Doesn't seem to realize he's just a cog in a large wheel 


Factor 4 stresses decisiveness, initiative, and decision-making, that is to say, for 
and the tendency to assume a dominant giving directions rather than executing 
role in leaderless situations. Here the them. The factor appears to conform to 
emphasis is on liking for responsibility traditional definitions of leadership. 


Factor 5. Personal Charm 


Ite А iteri TN 
йй ber Loading = os Descriptive phrase 
107 .878 .363 Always pleasant and cheerful 
71 .302 .171 Наза pleasant voice 
76 .282 .352 Easy to talk to 
176 .266 -460 Не is considered a “good sport" 
тог +251 .070 Easy going 
9 +242 -340 ‘Takes pride in his health 
19 +239 +064 Usually sees the humorous side of things 
80 +235 -472 Tends to see the good side of things 
57 — +253 —.341 Is inclined to be impatient with others 
93 —.235 —.376 Inclined to “рор off" on occasion 


These items define a pleasant, relaxed, dominant tone of cheerfulness and good 
approachable personality with a pre- humor. 
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Factor 6. Resource[ulness 


; Criterion -— 
иа. Loading cancer Descriptive phrase 
158 .899 .434 Able to by-pass red tape when necessary , 
68 .230 .627  Isfamiliar with main trouble spots of an airplane 
168 .226 ı647 Will really go to bat for things when he has to 
73 .222 .639 Usually finds some way to “crack a hard nut” 
51 .216 .388 His advice is asked by others 
45 .208 .649 Stimulates co-workers to think NS 
177 .200 .372 Teaches himself how to operate equipment without supervision 
33 .197 .721 Не learns new jobs quickly 
22 .183 -349 Has a good imagination 
135 —.187 —.574 Is helpless when working in awkward positions 


Ingenuity and resourcefulness appear ing ability, and an imaginative approach 
to form the central core of this factor, to problem-solving also associated. 
with good job knowledge, superior learn- 


Factor 7. Willingness and Adaptability 


tem ; iteri Б 
«ШЕ, Loading dsl Descriptive phrase 
140 .832 .so4 Моша be willing to begin in any kind of job to prove his ability 
115 ‚201 .665 Wants to adjust himself to new situations 
144 .186 -745 Helps others with their work whenever possible 
179 .176 .807 Is determined to make good 
107 .174 .363 Always pleasant and cheerful 
28 „172 .705 Usually tackles a job with enthusiasm 
119 .167 .611 Takes pride in group accomplishment 
102 .164 .69o Turns off power or motor equipment when not in use 
13 .160 “661 Сап be used effectively on several different types of work 
180 .183 .838 Не gives his best effort to whatever he is doing 


This factor emphasizes willingness, 
determination, and adaptability. The 
items portray the type of mechanic who 
wants to make good and prove his own 
capabilities. Eager to tackle new prob- 
lems, he has high morale and takes pride 
in group accomplishment. It is hardly 
surprising to find that such an individual 


often helps others. Though we cannot 
be sure as to the psychological reasons 
for this energetic service, one is inclined 
to speculate that the mechanics so de- 
scribed are by nature eager and com- 
pliant, and that it is this, rather than 
any overwhelming interest or ambition, 
which is responsible for their willingness- 
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Factor 8. Orderliness 


Item ; Criterion as 
number Leading correlation Descriptive phrase 
197 . 806 -516 Knows his own abilities well 
74 .217 -702 He accepts full responsibility for his own work 
159 .168 716 Usually assembles or disassembles parts in proper sequence 
73 .160 -659 Usually finds some way to “crack a hard nut" 
71 .158 .171 Has a pleasant voice 
147 .157 570 His quarters are usually neat 
137 = .166 —.437 15 careless of personal appearance 
10 —.165 —.564 Couldn't direct without irritating people 
54 = .154 —.467  Hesitates in making decisions on ordinary everyday questions 
135 —.150 —.574 Is helpless when working in awkward positions 


At first sight, the items appearing 
under this factor are much more hetero- 
geneous than those in the seven preced- 
ing it. We find here clustering together 
such varied characteristics as neatness, 
self-direction, insight, a pleasant voice, 
and accuracy in performance on the job. 
This diversity of item content can hardly 
be attributed to sampling error, as we 
find that factors with considerably less 
variance than this are nevertheless con- 
sistent and meaningful. We have ac- 
cordingly hypothesized that this factor 
defines a personality trait making for 
precision and exactness in thought and 


action. A mechanic having this trait 
knows his own mind and capacities, is 
neat and orderly in appearance, and 
takes a systematic approach to his work. 
He is methodical, but not without in- 
genuity, and can direct others without 
irritating them. In general, the impres- 
sion given is of an individual who is 
well organized but not compulsive, care- 
ful but not finicky. This factor is an in- 
teresting one whose constituent items 
tend for the most part to correlate rea- 
sonably well with our criterion of job 
Success. 


Factor 9. Ability to Motivate Others 


Item 


number Loading dd. Descriptive phrase 
171 ‚657 -715 Provides just the "spark" that is needed for effective teamwork 
183 ‚209 .613 Makes allowance for the limitations of others 
18 .204 -685 — Arouses ambition in fellow mechanics 
151 .192 -669 Не has made a good social adjustment on the job 
"m . 190 .649 Stimulates co-workers to think 
165 ‚187 .612 Quick to praise а man for a job well done 
83 .185 .641 Boosts the morale of any group in which he is working 
152 .170 -773 Respected by all who know him 
189 117$ .523 Heis good at organizing new activities 
67 —.170 —.795 Doesn't seem to realize he's just a cog in a large wheel 


Together these items describe the re- 
sponsible, task-oriented mechanic with 
a talent for group activity. His attitude 


toward the job and ability to organize 
make for effective teamwork among his 
co-workers. 


14 WRIGLEY, CHERRY, LEE, AND MC QUITTY 
Factor ro. Mechanical Proficiency 
Item Loadin Criterion Descriptive phrase 
number oading correlation scrip phres 

73 .707 .659 Usually finds some way to “crack a hard nut" 
34 .194 769 Uses good judgment in most situations , 

159 .185 .716 Usually assembles or disassembles parts in proper sequence 
61 .178 728 Не can organize his ideas effectively 
68 .176 627 Is familiar with main trouble spots of an airplane 
13 .172 66r Сап be used effectively on several different types of work 
33 .I59 721 Не learns new jobs quickly 

115 .158 .665 Wants to adjust himself to new situations 
15 — .186 —.683 Doesn't listen to suggestions 
95 —.172 —.848 Doesn't do his share of the work 


In this factor the primary emphasis is 
upon proficiency and the ability to cope 
with mechanical problems. However, the 
psychological nature of this proficiency 
is defined only in the broadest of terms. 
Reference is made to ingenuity and good 
judgment, to versatility, adaptiveness, 
and, indirectly, to job knowledge. Mo- 
tivational requirements are also men- 
tioned, as they were in Factor 1. Super- 
visors do not seem to make the distinc- 
tion between motivational and cognitive 
qualities which it is customary for psy- 
chologists to draw. 


Orthogonality of the square-root factors. Since 
square-root factors are orthogonal, it may at 
first sight seem strange that several factors may 
be extracted in what might appear to be the 
same area. For example, Factor 3, labeled "Ex- 
ecutive Ability" considers (among other things) 
ability to organize personnel, Factor 4, labeled 
"Leadership," stresses ability for giving direc- 
tions and assuming a dominant role, and Factor 
9, labeled “Ability to Motivate Others,” de- 
scribes a talent for group activity and for ef- 
fective teamwork with co-workers, Moreover, 
some items appear in two lists, viz., “Quick to 
praise a man for a job well done,” “An ex- 
tremely good organizer of personnel" "He is 
good at organizing new activities," “He doesn’t 
seem to realize he’s just a cog in a large wheel.” 

Nevertheless, each factor has a somewhat dif- 
ferent emphasis. Factor 3 stresses planning and 
organizational ability, Factor 4, leadership and 
the giving of orders, and Factor g, cooperation 
and sympathy for others. Most of the items in 
each list are peculiar to that factor. These 
items which appear in two lists may be pre- 
sumed to be factorially complex, embodying 


some element of one factor and some element 
of another, 

No doubt, by different placement of axes, 
some more comprehensive factor, extending 
over the entire area of personnel control and 
interpersonal relations, could be extracted, with 
subsidiary factors making the distinction Þe- 
tween organizational ability, leadership, and 
cooperation. Any such solution would certainly 
be acceptable. Yet the way the variance has 
been partitioned by the square-root method 
seems equally acceptable in indicating the prin- 
cipal conceptual distinctions made by the super- 
visors in their descriptions. 

An cven more striking instance of apparent 
overlapping is between Factor 5, labeled ''Per- 
sonal Charm," and Factor 13, labeled “Changea- 
bility of Mood.” Factor 13 not being reported 
in this monograph, it will suffice to note that 


the four items with highest loadings are: 
“Changeable in his attitudes," “Inclined to 
‘pop off’ on occasion,” “Gets mad if things 


don't go to suit him," and "He thinks he puts 
in too long hours." In contrast to the calm and 
relaxed personality of Factor 5, mechanics of 
the type portrayed by Factor 13 seem to be 
changeable, irritable, and given to outbursts of 
temper. But the cluster of negatively toned items 
in Factor 13 is not completely antithetical to 
the cluster of positively toned items in Factor 5- 
Factor 13 seems to be principally concerned 
with high emotionality, and Factor 5 with so- 
cial manner. In the correlation matrix, the 
correlations between the two sets of items ате 
well short of —1.00. This means that supervisors 
sometimes give the same response to both 4 
positively toned item and a negatively toned 
item which appears to be antithetical. This 
may sometimes be the result of carelessness in 
responding to the inventory, but it seems reason- 
able to believe that no man’s behavior is con- 
sistently good, poor, or average. This tendency 
to mark, say, “true” to antithetical phrases might 
on this line of reasoning be expected to be 
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especially characteristic of supervisors rating 
“average” mechanics. It may not be at all in- 
consistent for a supervisor to report a certain 
mechanic to be changeable in attitude, and yet 
to bc pleasant to talk to and to possess a pleasant 
voice. These negative interitem phi coefficients 
of much less than unity probably provide evi- 
dence that the halo effect was to some extent 
overcome by the raters. The fact that items of 
the type included in Factors 5 and 13 tend to 
have lower criterion correlations supports this 
explanation. The fact that it is often difficult in 
our language to find exact synonyms and exact 
antonyms is a further indication of the com- 
plexity of our behavior. 


V. RELATION OF THE FACTORS TO SUPER- 
VISORY RATINGS OF JOB PROFICIENCY 
In this section we consider: (a) the 

average criterion correlation for the ten 
most highly loaded items of each factor, 
(b) the pattern of job performance re- 
presented by items with high criterion 
correlations, and (c) the nature of the 
items which appear to be unrelated to 
ratings of either good or poor perform- 
ance. 

Average criterion correlations for the 
ten most highly loaded items for each 
factor. The criterion correlations will be 
recalled to be the phi coefficients between 
each item and the "'best-poorest" classifi- 
cation, with average mechanics excluded 
from the sample. Since these judgments 
of "best" and "poorest" were made by 
the same supervisors who provided the 
item ratings, they do not represent an 
independent measure of job proficiency. 
The criterion correlations may therefore 
be spuriously high, both for this reason 
and because they are based on the ex- 
tremes of the sample. The correlations 
are nevertheless useful for our present 
purpose, which is merely that of deter- 
mining the pattern of behavior which 
supervisors consider to be characteristic 
of good mechanics. E 

Table 2 presents the average criterion 
correlations for the ten items with high- 


TABLE 2 


AVERAGE CRITERION CORRELATIONS 
FOR THE FACTORS 


Average 
Lee ood Factor title criterion 
correlation 

1 General job efficiency .83 
то Mechanical proficiency 43 
3 Executive ability .70 
2 Social maladjustment —.68 
9 Ability to motivate others .67 
7 Willingness and adaptability .66 
6  Resourcefulness -58 
4 Leadership „Б 
8 Orderliness .54 
5 Personal charm +30 


Note—Correlations are based on descriptions 
of "best" and “poorest” mechanics. (‘‘Average”’ 
mechanics excluded.) 


est loadings on each factor. In the two 
previous studies (6, 7), factors dealing 
with practical efficiency, good judgment, 
foresight, etc., had been found more 
predictive than those relating to socia- 
bility, character, degree of emotionality, 
and other personality attributes. Results 
for the Survey Inventory agree with those 
already reported. The most predictive 
factor is the first; this, it will be recalled, 
stresses practical efficiency and good or- 
ganization. The least predictive factor is 
the one labeled “Personal Charm.” It 
seems to make rather little difference to 
quality of work whether or not a me- 
chanic is liked by his associates. 

The Survey Inventory factors are, on 
the whole, more predictive than those 
for the other two inventories. This dif- 
ference will be discussed in Section VI 
of this report. 

The pattern of performance indicated 
by items with large criterion correlations. 
Tables 3 and 4 list items with criterion 
correlations greater than or equal to .70. 
Items more commonly used to describe 
"best" mechanics appear in Table 3; 
those more often applied to "poorest" 
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TABLE 3 
CRITERION CORRELATIONS FOR ITEMS WITH HIGH POSITIVE PREDICTIVE VALUE (¢> .79) 


Item Correlation 
no. coefficient Item 
187 .870 Fits well into the organization 
116 .865 Goes over own work to see that it was done correctly 
8 .857 Seems to have a good future as a mechanic 
191 .846 Sets a good example through his habits of work 
180 .838 He gives his best effort to whatever he is doing 
91 .836 Very valuable іп а new operation 
92 .834 He should develop into one of the leading men in the crew 
163 .834 Handles additional and special responsibilities well 
161 .812 Works well without supervision 
169 .812 Works well without a lot of explaining 
174 . 811 Knows how to organize to get right things done at right time 
99 . 8o9 Is a better man than the average we have had in past years 
131 . 809 Sees what will be needed later and gets ready for it 
145 .808 Can take over іп ап emergency 
179 .807 Is determined to make good 
123 +799 On the lookout for better ways of doing the job 
III .797 He makes the best use of his ability 
6 .795 Shows good foresight in his planning 
106 .791 Very good at going ahead alone 
51 .788 His advice is asked by others 
200 .784 Is always a willing worker 
86 „975 Does jobs in order of importance 
152 -773 Respected by all who know him 
34 .769 Uses good judgment in most situations 
17 .740 Takes precautions to prevent damage to equipment 
190 .748 Makes the best of every bad situation 
144 +745 Helps others with their work whenever possible 
9o .731 Selects proper Tech. Orders and makes correct use of them 
41 .729 Has won nearly everybody's confidence 
188 «729 Would be very difficult to replace 
196 +729 Obtains co-operation from other people 
27 .728 Checks on the condition of the airplane and its equipment prior to flight 
61 .728 He can organize his ideas effectively 
154 .723 Has exceptional mechanical ability 
33 .721 He learns new jobs quickly А 
159 .716 Usually assembles or disassembles parts in proper sequence 
171 .715 Provides just the "spark" that is needed for effective teamwork 
70 .714 Makes plans fit in with those of fellow workers 
36 .710 Readily understands and carries out instructions 
28 ‚795 Usually tackles a job with enthusiasm 
74 „702 He accepts full responsibility for his own work 
24 .700 Knows how to put best foot forward 


= 


Note—Correlations are based on descriptions of "best" and “poorest” mechanics. 


mechanics excluded.) 


mechanics are presented in Table 4. The 
good mechanic will be seen to be de- 
scribed as: 


(a) cooperative—he gets on well in the organi- 
zation, is helpful to others, and succeeds in get- 
ting their cooperation; 

(b) keen to do his best—he approaches a job 
enthusiastically, does the best with it that he 
can, and wants to make good; 

(c) responsible—he works well without super- 
vision, can take over in an emergency, is good 


“Average” 


at going ahead by himself, and makes a succes? 
of any special responsibilities; 

(d) versatile and flexible in his ways—he 
watches for better ways of doing the job, an 
is useful in new operations; T 

(є) accurate—he makes sure that his work * 
correct; 

(f) foresightful with a good sense of propor 
tion—he carries out jobs in the right order e 
importance, and gets ready for what is needed: 


Some items in the list consider the 
ability level of the mechanic. The goot 
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TABLE 4 
CRITERION CORRELATIONS FOR ITEMS WITH H1GH NEGATIVE PREDICTIVE VALUE 


Item 


Correlation 

no. coefficient Item 

05 — .848 Doesn't do his share of the work 

6s —.824 Takes far too many breaks 

114 — .806 Loses too much time in waste motion 
170 —.700 Never has things ready when needed 

66 —.798 Seldom plans work ahead 

88 —.797 Never notices things to be done . 

67 —.795 Doesn't seem to realize he's just a cog in a large wheel 
121 —.795 Always ready with an excuse J 
146 —.78o Wastes time walking around and talking to others 
53 —.761 He abuses privileges 

70 —.759 His work requires more than a normal check-up 
138 —.750 Does not take important matters seriously 
192 —.750 Can't seem to make ends meet 

"m —.756 Thinks mostly of himself and not of other crew members 
130 —.752 Says one thing and does another 

148 —.736 Exaggerates too much 

1 —.720 Is frequently late for work 

156 —.728 Is a routine worker, doing only what is required 
109 —.718 Has a poor attitude 

87 —.717 Unfairly criticizes policies of his superiors 
186 —.716 Wastes fuels, lubricants and other supplies 
118 —.700 Often picks up any old wrench to do a job 

105 —.707 He criticizes men who are better than he is 

167 —.706 Gripes about the least thing 

112 — +703 Uses a “line” to try to impress others 


Note—Correlations are based on descriptions of "best" and "poorest" mechanics. (“Average 


mechanics excluded.) 


mechanic is described as having high 
mechanical aptitude and being able to 
learn new jobs rapidly. There is specific 
reference to selecting appropriate Tech- 
nical Orders and to assembling parts in 
proper sequence. Other items stress the 
practical application of knowledge and 
abilities. 

It is interesting to notice that the good 
mechanic is rather generally respected 
for his work. We are told that he wins 
nearly everybody's confidence, and that 
others go to him for advice. Other items 
in the list describe successful job per- 
formance in such general terms that we 
are supplied with no new information 
as to its specific nature. Examples of such 
items (which are nevertheless highly pre- 
dictive) are “Не seems to have a good 
future as a mechanic" and "He would 
be very difficult to replace." 


The items with high negative co- 
efficients, i.e., those which are more char- 
acteristic of poor mechanics, reinforce 
the above picture. The poor mechanic 
is portrayed as lazy, careless, and unco- 
operative. He has low morale, fails to 
plan ahead, and is never ready on time. 
Untrustworthy and inefficient, he says 
one thing and does another, and loses 
time in waste motion. Perhaps most sig- 
nificantly of all, he is described as a 
routine worker who does only what is 
required and fails to take important 
matters seriously. 

Items of low predictive value. Table 5 
lists items with criterion correlations 
lower than .20. As in the earlier studies, 
the social personality of the mechanic 
bears little relation to the rating he is 
given by his supervisor. It matters little 
whether he is easygoing and at ease in 
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TABLE 5 
CRITERION CORRELATIONS FOR ITEMS WITH Low PREDICTIVE VALUE (¢ < .20 
Item Correlation Fem 
no. coefficient 
8т .064 Rubs people the wrong way 
тот .070 Easy going 
164 .I21 Has a bold, direct manner 
71 .17ї Has а pleasant voice | 
16 . 18o Is at ease in any situation E 
35 — .007 Often uses gestures to put across his ideas 
100 — .046 Lets others take advantage of him 
149 — .046 Is always asking for advice " 
185 — .088 Gets resentful if property or other rights are trespassed on 
153 —.183 Worries over possible misfortunes 
160 —.183 Fears criticism from superiors 


Note—Correlations are based on “best” 
cluded.) 


any social situation or is irritable and 
liable to worries and fears. 


VI. Comparison or THREE 'ТҮРЕ$ 
or RATING ITEM 


In the research series of which this 
study is a part, factor analyses were made 
of items drawn from three sources. The 
Descriptive Inventory (7) was derived 
from supervisors interview descriptions 
of aircraft mechanics, the Factorial In- 
ventory (6) from factor-analytic studies 
of personality and ability, and the Survey 
Inventory, reported here, from rating 
scales and questionnaires designed to 
measure job performance in various 
fields. In this section, results from the 
three studies will be compared. 

The three inventories were adminis- 
tered to different samples of supervisors 
of aircraft mechanics under similar ex- 


and "poorest" mechanics. (" Average" 


mechanics ex- 


perimental conditions. The only material 


change was a decreased proportion of 
"average" ratings, relative to "best" and 
"poorest," in the Survey Inventory. The 
number and proportion of ratings of 
each type in each study are presented in 
Table 6. The increased proportion of 
extreme ratings in the Survey Inventory 
does not affect the criterion correlations, 
since they are based upon "best" and 
"poorest" ratings only. However, in the 
factor analyses there might be a greater 
concentration of item variance in the 
early factors of the Survey Inventory. 
This possibility should be kept in mind 
in the comparisons which follow. 

Table 7 compares (a) the distribution 
of variance and (b) the average criterion 
correlation, for the first 10 factors of the 
three inventories. If, for the rating of 
job performance, we were obliged to re- 


TABLE 6 
CoMPARISON OF SAMPLES IN THE THREE INVENTORIES 


Number of ratings 


Percentage of ratings 


Inventory 
Best Average Poorest Total Best Average Poorest 
Descriptive 112 224 92 428 26.2 52.3 21.5 
Factorial 95 194 94 383 24.8 50.7 24.5 
Survey 148 179 137 464 31.9 38.6 29.5 
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TABLE 7 


COMPARISON OF THE DISTRIBUTION OF VARIANCE 
AND OF THE AVERAGE CRITERION CORRELATION IN 
THE Factors OF THE THREE INVENTORIES 


Contribution Average 
to criterion 
Taverner: variance correlation 
Factor Factors Factor Factors 
I 2-10 I 2-IO 
per per 
cent cent 
Descriptive 20.6 15.3 .74 -49 
Factorial 9.0 23.4 41 .4ї 
Survey 26.8 17.0 .83 .бо 


Note—Correlations are based on “best” and 


"poorest" mechanics. (“Ауегасе" mechanics ex- 
cluded.) 


strict ourselves to items drawn from a 
single factor of a single inventory, the 
first factor of the Survey Inventory would 
have to be our choice. It defines better 
than any other the qualities considered 
by supervisors to be characteristic of a 
good mechanic. The criterion correla- 
tions are higher, and more item variance 
is accounted for, than for any other 
factor. 

A rather different pattern emerges if 
the results for Factors 2-10 of each study 
are considered. The Factorial Inventory 
is seen to be more comprehensive in 
coverage than the Survey Inventory, but 
its items tend to be less job-centered and 
less predictive. The Descriptive Inven- 
tory falls between the other two in re- 
spect to predictive power. | 

The different areas of behavior de- 
scribed by each inventory. Differences in 
the subject matter of the inventories may 
perhaps be summarized most readily by 
making an a priori classification of the 
factors, and then attempting to deter- 
mine the extent to which each behavioral 
area is represented in each inventory. 
The statistical orthogonality of the 
square-root factors need not preclude us 


from doing this, since the presence of 
factorially complex items in several fac- 
tors indicates their semantic interrelated- 
ness. (See the discussion of orthogonality 
and overlapping of factors in the later 
part of Section IV.) 

Because of the multiplicity of factors, 
the listing will be restricted to the ten 
largest factors in each study. Table 8 
presents this classification. The grouping 
of factors is to some extent arbitrary, and 
alternative classifications are possible. 
Results are so clear, however, as to leave 
little doubt as to the nature of the differ- 
ences in content. 

1. Practical ability. By “practical abil- 
ity,” we mean the capacity to make de- 
cisions quickly, carry out jobs in order 
of importance, find better ways of doing 
a job, explain matters adequately, or- 
ganize well, and plan work in advance. 
This aspect of job proficiency takes a 
central position in each inventory. Seven 
factors in this area appear in Table 8, 
and criterion correlations are in most 
cases high. 

2. Sense of responsibility toward work. 
There are four factors in this area in the 
Descriptive Inventory, and one in the 
Survey Inventory. A mechanic's willing- 
ness and dependability seem to be the 
aspect of job behavior which is discussed 
at greatest length by the supervisors. As 
might be predicted, criterion correlations 
in this area are high. 

3. Pattern of interests. Interest in the 
job is not well represented in any of the 
three inventories. The Descriptive In- 
ventory contains a factor entitled “In- 
terest in Aircraft Maintenance,” and the 
Survey Inventory factor of “Willingness 
and Adaptability” might perhaps also 
have been included in this grouping. In 
both factors, the items are rather broad 
and nonspecific. The Factorial Inventory 
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TABLE 8 
COMPARISON OF THE TEN MAJOR FACTORS IN THE DESCRIPTIVE, FACTORIAL, 


AND SURVEY INVE) 


TORIES 


Average 


Area of behavior Factor criterion Factor title 
number er 
correlation 
1. Practical ability D6 —.71 Failure to use knowledge effectively 
D; 44% Teaching capacity 
Ез — 71 Tendency to indecision 
F4 .38 Pract ability 
51 .83 General job efficiency 
S3 -68 Executive ability 
Sro 71 Mechanical proficiency 
2. Sense of responsibility towards work Dr 74 Sense of responsibility 
D3 .so Willingness for work 
D4 —.so Laziness and lack of initiative 
D8 .58  Industriousness 
S; .66 Willingness and adaptability 
3. Pattern of interests D2 дт Interest in aircraft maintenance 
F6 .48 Cultural interests 
4. Intellectual ability Do -55 Memory 
5. Personality characteristics in the job 
situation Fs Lack of cooperation 
S6 Resourcefulness 
S8 Orderliness 
S4 Leadership 
So Abilitv to motivate others 
6. Social adjustment F2 Social immaturity 
F7 Shyness 
ЕВ Sociability 
Fo Submissiveness 
S2 Social maladjustment 
Ss Personal charm 
7. Emotional adjustment Fr Tendency to neuroticism 
Ето Emotional sensitivity 
8. Character Ds Weakness of character 
Dio Self-control 


Notes—r. Correlations are based on "best" and 
cluded.) 


2. D, F, and S indicate that the factors are from the Descriptive, Factorial, and Survey Inven 


respectively. 


contributes a factor of "Cultural In- 
terests" to this area. This serves to indi- 
cate the different and much less job- 
relevant type of item utilized by factor- 
analysts in their studies of personality 
and ability. 

4. Mental abilities. Only slight refer- 
ence is made to ability and to job knowl- 
edge. In the report on the Descriptive 
Inventory (7, p. 232), we commented on 
the fact that supervisors stressed motiva- 
tion and practical efficiency, so that no 
clearly defined factors of memory, rea- 


soning, and problem-solving ability 


“poorest” mechanics. ("Average" mechanics ех” 


tories 


emerged. Items were included in the 
Factorial Inventory to assess mental abili- 
ties. Factors of creativity, mental alert- 
ness, etc., emerged, and the results leave 
no doubt that all-round intellectual dull- 
ness is a variable contributing to poo" 
performance on the job. However, these 
factors all account for a rather smal! 
proportion of the variance. This means 
that the differences between mechanics 
are mostly described in such a way aS to 
be attributable to factors of a more 
practical kind. Items dealing with sharp” 
ness of mind, ability to learn quickly: 
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etc., tend to appear along with items de- 
scribing general job efficiency, organiza- 
tional capacity, and the like, rather than 
by themselves. In no study is there ex- 
tended reference to a mechanic’s knowl- 
edge of his job. 

5. Personality characteristics in the job 
situation. Personality traits seem to di- 
vide into two groups: (а) those which are 
relevant to the job situation, such as 
being orderly, resourceful, and able to 
motivate others; (b) those which are only 
slightly or not at all relevant in the job 
situation, such as degree of shyness. 
There are five factors under the heading 
of "Personality characteristics in the job 
situation." Four of these are from the 
Survey Inventory. Probably because the 
items for this inventory were written 
with specific reference to the job situa- 
tion, the stress in the personality-type 
items is upon degree of cooperativeness, 
ability as a leader, etc. 

6. Social adjustment. The Factorial 
Inventory items deal to a greater extent 
than the others with the entire pattern 
of a man's interpersonal relationships, 
and his ability to manage adequately in 
social situations. Four of the six factors 
in this area appear in the Factorial In- 
ventory. Criterion correlations are in 
general lower for these factors. The shy 
and introverted mechanic seems to be 
quite acceptable. On the other hand, the 
hostile and trouble-making mechanic is 
not. 

7. Emotional adjustment. The two 
factors in this area are, as might be ex- 
pected, from the Factorial Inventory. 

8. Character. The Descriptive Inven- 
tory makes greater reference than any 
other to defects of character—heavy 
drinking, irresponsibility in money mat- 
ters, and the like. Criterion correlations 
are, as a rule, rather low. 


From Table 8, a summary statement 
can be made of the distinctive char- 
acteristics of each of the three inven- 
tories. 

The Descriptive Inventory (phrases 
supplied by supervisors) stresses a me- 
chanic’s willingness for work, depend- 
ability, sense of responsibility. 
Furthermore, there is more in this in- 
ventory than in either of the others 
about a mechanic's moral habits—i.e., 
about his monetary and drinking habits, 
his trustworthiness, his punctuality, etc. 
These are probably areas of behavior 
where there are likely to be disputes be- 
tween supervisor and mechanic, so that 
any deficiences are strongly impressed 
upon the supervisor's attention. The 
supervisors tend to use evaluative rather 
than analytical descriptions. 

The  Factorial Inventory (phrases 
drawn from factor-analytic studies re- 
ported in the literature) stresses the me- 
chanic’s social and emotional adjust- 
ment—i.e., his emotional sensitivity and 
variability, his shyness, tendencies to sub- 
missiveness, etc. In general, these aspects 
of personality are not very closely re- 
lated to job performance. The criterion 
correlations are lower than for any other 
area of behavior. This finding must be 
qualified in one important respect. There 
are factors of "social immaturity" and 
"social maladjustment" which portray 
"difficult" and somewhat antisocial per- 
sonalities (as opposed to merely shy and 
submissive individuals), and these me- 
chanics are rated poor. In addition, those 
factors which relate more directly to 
social behavior on the job itself (e.g. 
leadership ability, ability to obtain co- 
operation from others) have rather high 
criterion correlations. 

The Survey Inventory (phrases ob- 
tained from researchers in applied psy- 


and 


—- 266 
e 


f. 
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chology) stresses practical ability. It also 
yields factors which describe what we 
have called ‘Personality characteristics 
of the job situation,” viz., resourcefulness 
and orderliness. Its factors of social ad- 
justment are rather strongly job oriented. 
Motivation is stressed less in the Survey 
Inventory than in the Descriptive In- 
ventory, but there is greater emphasis 
upon efficiency and good interpersonal 
relations. The Survey Inventory items 
are less evaluative but more analytic than 
those of the Descriptive Inventory, and 
more job centered than those of either 
of the other inventories. 

This greater “job-centeredness” of the 
Survey Inventory is reflected in its ap- 
preciably higher criterion correlations. 
Thirty-three per cent of the Survey In- 
ventory criterion correlations are greater 
than .70. Only 18 per cent of the Descrip- 
tive Inventory items and 10 per cent of 
the Factorial Inventory items are as high 
as this. Hence, if we had to confine our- 
selves to any single one of the three in- 
ventories, the Survey Inventory would 
be the most reasonable choice. 

The over-all picture of the good me- 
chanic presented by the three studies is 
that of a man who has in mind the needs 
and demands of his job, who is willing 
to work hard, and who is vigorous and 
mentally alert, displaying good judg- 
ment and powers of decision. He is pre- 
cise, orderly, and dependable; yet at the 
same time he displays initiative and acts 
for himself. He is efficient and practical. 
More is said about the use he makes of 
his abilities than about the abilities 
themselves. The qualities which have 
been listed are probably useful assets in 
any form of work. Little is said about 
the special features of a mechanic’s job 
situation which distinguish it from other 
types of work. 


One of the important determinants of 
quality of performance on the job seems 
to lie in differing standards of value: we 
can differentiate between the mechanic 
who cares about his work, is ambitious, 
and wants to get on, and the one whose 
interest lies in getting privileges and 
time off and avoiding work. A second 
main line of distinction is between the 
mechanic who is systematic and orderly 
and his associate who is inefficient and 
lacking in common sense. In general, 
these items are strongly job-centered, and 
the occupational characteristics which 
they describe are not always easily ех 
pressed in the standard terms used lor 
accounts of ability and personality 
structuring. 

The dependence of factorial results 
upon the sampling of variables. The 16 
sults of the three studies show Ve"! 
clearly how content differs from опе in 
ventory to another. Obviously selectio? 
of items in factorial studies of this tyP^ 
is a matter of the utmost experiment? 
importance. Before any final and defin 
tive factorial description of the area can 
be provided, some logical definition А 
needed of a population of items from 
which random samples of items may 
drawn. Once the population of variables 
can be determined, then the correlation? 
configuration will tend to be similar from 
one factorial study to another (provide 
the sample of variables was large an 
the persons tested in the various studie 
also formed random samples from 2 de 
fined population). Given this similarit? 
of the correlational configuration, the 
clusters obtained from one rando? 
sample of items will then tend to agree 
with the clusters obtained from another 
random sample. 

The problem then is in the definitio? 
of the population of variables. There 
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seem to be at least two requirements: 
(a) to establish the possibility of enu- 
meration of all members of the popula- 
tion or of an unbiased and sufficiently 
large proportion of the members of the 
population to provide a basis for ran- 
dom sampling, (b) to demonstrate that 
the population so defined provides ade- 
quate representation of the area of be- 
havior to be investigated. 

Whenever we are operating with en- 
vironmental givens (i.e., variables already 
present in the psychological environ- 
ment, as opposed to artifacts constructed 
at will by the investigator), it is gener- 
ally not difficult to formulate some cri- 
terion for inclusion in the population, 
and accordingly to devise some random 
basis for sampling. For example, current 
semantic usage can provide a basis for 
sampling. The complete set of adjectives 
appearing in an English dictionary can 
be listed, as has been done by Allport 
and Odbert (1), and the sample drawn 
from these. Cattell (3, p. 216) prepared 
a comprehensive list of “basic trait ele- 
ments" in this way. Likewise environ- 
mental situations can be sampled. Bruns- 
wik and his co-workers (2, pp. 41-46), 
after protesting that many laboratory 
experiments use unimportant variables, 
unrepresentative of the environment and 
interlocked in some artificial way, have 
made effective use of sampling various 
environmental situations in their studies 
of size constancy. А 

Our procedure in selecting items for 
the three inventories was somewhat simi- 
lar. For the Descriptive Inventory (7) 
phrases were sampled from the descrip- 
tions of mechanics made by their super- 
visors. The population in this case con- 
sists of the various phrases currently 
used by Air Force supervisors to describe 
the mechanics under their control. The 


Factorial Inventory (6) was developed 
from the population of items used in 
published factor analyses in the area of 
personality, and the Survey Inventory 
from the population of published items 
in rating scales and questionnaires in 
industrial psychology. 

The principal difficulty is in meeting 
the second requirement for a population 
of variables. No one of our three pro- 
posed definitions of the population of 
job-knowledge descriptions proved to be 
sufficiently broad to subsume all aspects 
of job performance. For’ example, rea- 
sons for interest or disinterest in work 
are not adequately elicited by any in- 
ventory. Likewise, no one provides a 
sufficiently explicit account of the role 
played by job knowledge in job perfor- 
mance. To study these areas, further 
items would have to be prepared. Evi- 
dently the task of item-writing is not yet 
at an end, despite the large number of 
scales prepared by psychologists during 
the last fifty years. 

This appearance of precise definition 
of the population may be illusory even 
when operating with the population of 
dictionary adjectives, as Cattell did. 
Many adjectives have various shades of 
meaning, and often no instructions are 
given to the subjects requiring them to 
select one of these meanings rather than 
any other. Indeed, meaning can gener- 
ally be better indicated by phrases, which 
provide a context, than by words in isola- 
tion. 

Once, however, we are obliged to con- 
clude that there is a place for further 


° The sampling in this latter case was modified 
by the attempt to tap previously unrepresented 
areas of behavior, but the general argument is 
not thereby affected, since further inventories 
similar to the Survey Inventory could obviously 
be obtained from the original collection of items. 


24 WRIGLEY, CHERRY, LEE, AND MC QUITTY 


item-writing by the investigator, then it 
follows that we are at present only 
partially able to achieve a definition of 
a population of items. So long as the 
investigator is free to create new items 
at will, it seems to be premature to try 
to provide a rigorous delimitation of the 
population of items from which a sample 
is to be drawn. 

The function of the factor analysis. 
Because we are not yet in a position to 
define sampling procedure exactly, the 
development of a satisfactory rating scale 
for job performance has therefore to de- 
pend upon the progressive selection, re- 
finement, and supplementation of items. 
The principal function of the factor 
analysis reported here has been to pro- 
vide an empirical basis for preparing a 
briefer scale. The interview situation can 
then be used to assist in elucidation of 
the factors represented therein. After de- 
scribing a mechanic in terms of the items, 
a supervisor can be asked further ques- 
tions, e.g., with respect to the reasons for 
unusually high motivation, whether it 
indicates a man’s interest in his own ad- 
vancement, or his liking for mechanical 
assignments, or a favorable attitude to- 
ward the Air Force and its organization. 
These supplemental questions can be 
framed much more appropriately around 
the structure of the factors than with re- 
spect to individual items. The factor 
analyses, therefore, represent an initial 
step toward the better definition of the 
population of items for the rating of 
mechanics’ performance. 


VII. SUMMARY 


‘The research here reported is the last 
of three studies designed to investigate 
the different psychological correlates of 
job proficiency in the field of aircraft 
mechanics. Inventory items hypothesized 
to be related to job performance in this 


field assembled from various 
sources, and supervisors were required to 
describe mechanics in terms of these 
phrases. This report considers the fac 
torial structure of the items, and the re- 
lation of each item to the supervisors 
over-all ratings of the mechanics. 

The report is also designed to provide 
an example of application of the square 
root method of factor analysis, as modi- 
fied by Wrigley and McQuitty (9). The 
method has been devised for use with 
correlation matrices which are too large 
to be analyzed by standard procedures- 
There is a brief outline of the criticisms 
generally made of the square-root 
method, the modifications intro 
duced in an attempt to overcome these 
deficiencies. А 

Items in present study were systematik 
cally drawn from a set of rating scales and 
questionnaires which had been designed 
by various psychologists to measure dif- 
ferent aspects of job proficiency. тре 
items in the initial sample were revised 
in accordance with the results of a pr€ 
liminary administration. The revise“ 
scale of зоо items, known as the Survey 
Inventory, was then administered to 461 
supervisors of Air Force Aircraft and 
Engine Mechanics. They were requiret 
to describe a "best," “poorest,” or "aver" 
age" mechanic of their own choosing 1” 
terms of "true-false" responses to the 
items. 

A square-root factor analysis was the? 
made of the correlations between thes? 
item responses. The ten leading factors 
were named: (a) General Job Efficiency” 
(b) Social Maladjustment, (c) Executiv? 
Ability, (d) Leadership, (e) Person? 
Charm, (f) Resourcefulness, (g) willing 
ness and Adaptability, (/) Orderlines* 
(i) Ability to Motivate Others, (j) M* 
chanical Proficiency. 

Phi coefficients were also calculated РЁ 


were 


and 
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tween the inventory items and the “best- 
poorest” dichotomy. These correlations 
indicate the factors and the individual 
items which supervisors judge to be most 
relevant to job proficiency, and thereby 
serve to define more precisely super- 
visors’ conceptions of the nature of good 
and poor job performance. 

These results were then compared 
with those obtained from the analysis of 
(a) the Descriptive Inventory (7) (where 
items were derived from relatively un- 
structured descriptions of "best" and 
“poorest” mechanics by their supervi- 
sors) and (b) the Factorial Inventory (6) 
(where items were derived from factor- 
analytic studies). The inventories agree 
in the findings that practical capabilities 
are more necessary in a successful me- 
chanic than general intellectual ability, 
and that lack of motivation and a poor 
sense of responsibility are more detri- 


mental than a poor level of socioemo- 
tional adjustment. Each inventory tends, 
however, to have a distinctive coverage 
of its own. The Descriptive Inventory 
stresses a mechanic’s willingness for work 
and his dependability, the Factorial In- 
ventory his social and emotional adjust- 
ment, and the Survey Inventory, his 
practical ability. The items of the Survey 
Inventory are shown to be of broader 
coverage and somewhat more predictive 
and job-centered than those in the two 
preceding inventories. Hence the Survey 
Inventory would be the most reasonable 
choice if a single one of the three were 
to be selected for assessing job profi- 
ciency. Because of the distinctive nature 
of each inventory, however, a shortened 
scale may be expected to be a better 
measuring instrument if based on all 
three inventories than if developed from 
any single one. 


APPENDIX 


SURVEY INVENTORY FOR AIR FORCE MAINTENANCE PERSONNEL 


PURPOSE 
This inventory is part of a rescarch study on 
the attitudes and work habits of Airplane and 
Engine Mechanics. Your assistance is needed in 
carrying out this research because of your ex- 
perience and close association with men in the 
maintenance field. 


INSTRUCTIONS 


Think back over the crews of Airplane and 
Engine Mechanics you have worked with during 
the last two years. Think of a person who is, in 
your opinion, just an AVERAGE® mechanic; 
that is, a mechanic who is neither especially 
good, nor especially poor. Do not pick out a 
crew chief or supervisor when you make this 
selection, А 

Statements are listed on the following pages 
which will help you describe the mechanic you 


1 Other supervisors received directions to de- 
scribe the "Best" mechanic they had known 
during the previous two years, and still others, 
to describe the “Poorest” mechanic. 


now have in mind. Say each statement to your- 
self and decide whether or not it fits this 
mechanic, If it does, circle the T (for true) at 
the left of the statement; if not, circle the F 
(for false). 

Decide on each statement before going on to 
the next. Guess whenever you are not certain 
whether or not a statement describes the man. 
Ве sure to answer all statements as either true 
or false. 

NOTE: All information will be held in strict- 
est confidence and used for research purposes 
only. It WILL NOT affect you or the mechanic 
in any way. 


SURVEY INVENTORY 


Y E 1, Is frequently late for work 

T F 2. Is unwilling to lend a helping hand 
T F 3. Hasa very sharp mind 

T F 4. Likes to make decisions 

T E 5. Talks a lot to everybody 

T F 6. Shows good foresight in his planning 
T Е 7. Lacks snap in getting the work done 
E FEF 8. Seems to have a good future as a 


mechanic 
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. Takes pride in his health 
. Couldn't 


direct without irritating 


people 


. He remains calm under pressure 
. Has slow reactions 
. Can be used effectively on several 


different types of work 


. Lacks confidence in own opinions 

. Doesn't listen to suggestions 

. Is at ease in any situation 

- Takes precautions to prevent damage 


to equipment 


ambition in fellow me- 


chanics 


. Usually secs the humorous side of 


things 


. He thinks he puts in too long hours 
. Likes to take on responsibilities 

. Has a good imagination 

. Criticizes co-workers 

. Knows how to put best foot forward 


. Is excellent for instructing beginners 
. His 


personal 
criticized 


conduct cannot be 


. Checks on the condition of the air- 


plane and its equipment prior to 
flight 


28. Usually tackles a job with enthu- 


siasm 


. Conforms to regulations and policies 
. Does not improve with experience 

. Sometimes acts childish 

. Never quits ahead of time 

. He learns new jobs quickly 

. Uses good judgment in most situa- 


tions 


. Often uses gestures to put across his 


ideas 


. Readily understands and carries out 


instructions 
reached limit of his 
abilities A 


. Will trample over anyone to benefit 


himself 


. Usually quick and accurate in emer- 


gencies 


. Has poor posture 


Has won nearly everybody's confi- 
dence 


Often takes rumors seriously 


. Plenty of military snap and bearing 
‚ Doesn't clean tools before putting 


them away 


. Stimulates co-workers to think 
. Can stand kidding 
. Thinks mostly of himself and not 


of other crew members 


. Does very little original thinking 
. Seeks 


and does additional 


tasks 
beyond those required 
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. Is inclined to be impatient 


. Lets difficulties get 1 а 
. He makes definite decisions and de 


. Is familiar with main troubl 
. He thinks his supervisor 


. Makes plans fit in 


. Easy to talk to 
‚ Gets mad if things don't go to 


. Quick to size up a situation 
. His work requires more than а 


. Is very selfish i 
. Uses correct torque on bolts and ¥ 
. Does jobs in order of importance " 


. Never notices things to be done 


. Selects 


. Makes promises he doesn't kecp 
. His advice is asked by others 


Apparently not physically fit 


. He abuses privileges 
. Hesitates in making decisions on 


ordinary everyday questions 


5. Frequently gives short answers 
. Has trouble adjusting himself 10 


changed conditions R 
with 


others 


. Uses parts catalogue and stock list 


correctly in ordering parts 
him down 


fends them with courage 


. He can organize his ideas effectively 
2. Expects too rapid advancement њу 
‚ He is apt to be master of a situatio 


if it needs a leader 


. Needs to be prodded occasionally 
. Takes far too many breaks 

. Seldom plans work ahead ; 

. Doesn't seem to realize he's Jus 


i; 


in a large wheel | 
СЕ 8 e spols 
of an airplane 
P s are (00 
strict 
with those of 
other fellow workers 


. Has a pleasant voice 
. Forgets to return borrowed tools 
. Usually finds some way to 


"crack ® 
hard nut" 

He accepts full responsibilty for 
own work 

Allows his emotions to color 
judgment 


his 


his 


suit 
him 

nor- 
mal check-up 


. Tends to sce the good side of thing? 
. Rubs people the wrong way 

. He seldoms asks for any time off i 
. Boosts the morale of any grouP 


in 
which he is working 


uts 


Unfairly criticizes policies of 
superiors 

" nv 
Feels that he owes nothing to A" 
body d 

з! 

proper Tech. Orders а! 
makes correct use of them 
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. Very valuable in а new operation 

. He should develop into one of the 
leading men of the crew 

. Inclined to “pop off" on occasion 

Risks getting in bad with superiors 

to stand up for his fellow workers 

. Doesn't do his share of the work 

. Can't be told anything 

. Can talk intelligently on almost any 
topic 

. Cannot handle several details of his 
job at the same time 

. Is a better man than the average we 
have had in past ycars 

. Lets others take advantage of him 


. Easy going 


. Turns off power or motor equip- 
ment when not in use 


. Is usually dressed neatly 
. Gets help when in difficulties 


. He criticizes men who are better 


than he is 


. Very good at going ahead alonc 

. Always pleasant and cheerful 

. He always waits his turn 

. Has a poor attitude 

‚ Always shows proper respect toward 


superiors 


. He makes the best use of his ability 
. Uses a "line" to try to impress others 
. Resents being given a rush job 

. Loses too much time in waste mo- 


tion 


. Wants to adjust himself to new situ- 


ations 


- Goes over own work to see that it 


was done correctly 


. His appearance creates a distinctly 


favorable impression 


. Often picks up any old wrench to 


do a job 


. Takes pride in group accomplish- 


ment 


. Has "jumpy" nerves 
. Always ready with an excuse 
. Goes his own 


way regardless of 


others 


. On the lookout for better ways of 


doing the job 


124. Changeable in his attitudes 
125. Not as tactful in criticizing as he 


should be 


. Is not very sociable — 
27. Does everything possible to keep 


maintenance costs down 


. Avoids doing things which do not 


reflect credit on him 


. Makes mistakes in judging the char- 


acter and ability of others 


. Says опе thing and docs another 


T 
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131. 


132. 
. Never cocky or over-bearing 
. Plans ahead for almost any emer- 


Sees what will be needed later and 
gets ready for it 
Is a never-tiring worker 


gency 


. Is helpless when working in awk- 


ward positions 


‚ Dissatisfied with his present job 
- Is careless of personal appearance 
. Does not take important matters 


seriously 


. He butts in on others’ responsi 


bilities 


- Would be willing to begin in any 


kind of job to prove his ability 


. Resents criticisms or suggestions 
- Makes trouble over little things 
‚ Reports on time for scheduled ap- 


pointments, meetings, formations, 
etc. 


. Helps others with their work when- 


cver possible 


. Can take over іп an emergency 
. Wastes time walking around and 


talking to others 


- His quarters are usually neat 

. Exaggerates too much 

. Is always asking for advice 

. Always does things in one particular 


way 


. He has made a good social adjust- 


ment on the job 


. Respected by all who know him 
3. Worries over possible misfortunes 

. Has exceptional mechanical ability 
5. Often 


waits unnecessarily for di- 
rections 


. Is a routine worker, doing only what 


is required 


. He is quarrelsome 
. Able to by-pass red tape when ne- 


cessary 


. Usually assembles or disassembles 


parts in proper sequence 


. Fears criticism from superiors 
- Works well without supervision 
. He seems to know when to keep his 


mouth shut 


- Handles additional and special re- 


sponsibilities well 


. Has a bold, direct manner 
- Quick to praise a man for a job 


well done 


. Discusses plans frequently with su- 


perior 


- Gripes about the least thing 
168. 


Will really go to bat for things when 
he has to 


Lm 


2. 


3. 


4. 


5. 


б. 


WRIGLEY, CHERRY, LEE, AND MC QUITTY 
F 169. Works well without a lot of explain- TF 185. Gets resentful if property or other 
ing rights arc trespassed on 
F 170. Never has things rcady when needed ГЕ 186. Wastes fucl, lubricants, and other 
F 171. Provides just the "spark" that is supplies К 
needed for effective teamwork Г Е 187. Fits well into the organization 
F 172. A one-job man, no wide grasp of 'T F 188. Would be very difficult to replace 
maintenance T F 189. He is good at organizing new ac- 
F 173. Tries to flatter crew chicf i 
Е 174. Knows how to organize to get right T F 190. Makes the best of every bad situa- 
things done at right time tion " 
F 175. Thinks his work is too monotonous T Е 191. Sets a good example through his 
F 176. He is considered a “good sport” habits of work 
F 177. Teaches himself how to operate T F 192. Can't seem to make ends тесі 
equipment without supervision T 193. Unwilling to stick his neck out 
F 178. Complaints are made (more often Т Е 194. Inclined to pay much attention 10 
than average) on the quality of his aches and pains 
his work T F 195. He likes the way the maintenance 
F 179. Is determined to make good squadron is run 
Е 180. He gives his best effort to whatever T F 196. Obtains co-operation from other 
he is doing people 
Е 181, An extremely good organizer of per- Т Е 197. Knows his own abilities well В 
sonnel T F 198. He accepts decision of the majority 
F 182. Prefers to be a follower rather than without showing his displeasure 
a leader when the decision is against him 
Е 183. Makes allowance for the limitations Т Е 199. Fails to make a good first impression 
of others T F 200. Is always a willing worker 
F 184. Always wanting sympathy 
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The Effect of Varying Amounts and Kinds of Information 
As Guidance in Problem Solving 


BERNARD К. CORMAN? 


Michigan State University 


I. INTRODUCTION 


PROBLEM may be said to exist when 
habitual responses fail to lead to the 
attainment of a desired goal. A search 
for a new response must then be under- 
taken. In intellectual problems, the 
problem solver must not only search for 
possible alternative courses of action, but 
must also select from among these al- 
ternatives the ones that will most success- 
fully remove the obstacle to the goal. 
The search, then, is for information that 
will give structure to the problem. Pre- 
sumably, as the amount of information 
developed is increased, the necessity for 
search is reduced. And, when all relevant 
information for the problem is available 
and understood, a problem no longer 
exists, although practice may still be re- 
quired to make the new response habit- 
ual. 
Information may be developed by the 
problem solver’s own motivated search, 
or it may be made available to him. In- 


sion of a dissertation sub- 
mitted in 1954 to the faculty of Teachers Col- 
lege, Columbia University, in partial fulfillment 
of the requirements for the degree of Doctor of 
Philosophy. 
The writer is 
Lorge, who directe 


1 This study is a revi 


deeply indebted to Dr. Irving 
d the dissertation, for his 
sustained interest, encouragement, and stimulat- 
ing counsel. The writer also wishes to express 
his appreciation to Drs. Robert Thorndike, Her- 
bert Solomon, and Lincoln Moses for their many 
helpful suggestions. 


formation supplied by directions, by 
clues, or by other guidance, however, 
may vary in amount. There may be a 
greater or lesser reduction in the num- 
ber of alternatives which remain to be 
discriminated. 

If little or no information is supplied, 
the problem solver may fail to develop 
the primary information needed. This 
failure, in turn, may lead to frustration 
and a turning away from the problem. 
It has been shown that, left to their own 
devices, problem solvers often set up 
false assumptions which make for un- 
necessary restrictions that delay or pre- 
vent solution (14). Those who argue the 
efficacy of specifically directive guidance, 
point to the large amount of time ex- 
pended in the discovery of the required 
information, with the important risk of 
the development and perseveration of 
errors (21, р. 147). 

On the other hand, by presenting large 
and significant amounts of information, 
the necessity for search on the part of 
the problem solver may be lessened. The 
opportunity for a thorough examination 
of the problem may thereby be re- 
stricted. Situations have been described 
in which specific directive infermation 
was accompanied by failure .^ solve 
problems like those on which the guid- 
ance had been given (9, 12, 23). This lack 
of transfer has been explained as a fail- 


ure of the solver to become cognizant of 
5 aa 
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relationships among the structural ele- 
ments of the problems studied. Informa- 
tion supplied in guidance, it has been 
suggested, should provide an opportu- 
nity for discovery of these relationships 
by the problem solver through his own 
motivated efforts, rather than anticipate 
such discovery by the identification of 
specific methods or principles (9). While 
in “discovery” more errors may occur, it 
has been felt that, in the context of a 
more intense search, such errors would 
not be deleterious (19). 

Faced with two seemingly contradic- 
tory points of view, those charged with 
the guidance of learning experiences in 
problem solving are asked to choose be- 
tween two unattractive alternatives in 
determining how much information to 
provide. It is the purpose of this mono- 
graph to re-examine the effects of giving 
varying amounts of information in the 
guidance of problem solving. 


II. RATIONALE FOR AN EXPERIMENT 


If problem solving (as distinguished 
from routine drill) involves the search 
for a new response, then some “discov- 
ery" will accompany the successful solu- 
tion of any problem. As Stroud has ob- 
served: 

The сазе for discovery is difficult to evaluate, 
and indeed cannot be evaluated until the term 
18 given precise definition. There is discovery 
with help, much or little, and discovery without 
help. In a sense any act of understanding is a 
discovery no matter by how much help or ex- 


planation the understanding is achieved 
(20, pp. 582-583). 


It follows that, while the extent of 
search may be more or less restricted, 
any test of the effectiveness of varying 
amounts of information given as guid- 
ance would require that, as an experi- 
mental condition, all subjects (Ss) be en- 
gaged in discovering a correct response. 


In many of the experiments which have 
been taken as supporting the desirabil- 
ity of methods of tuition emphasizing 
"discovery" the comparisons have been 
between groups who have been shown 
a correct solution and groups who must 
discover such a solution (9, 13, 19). Such 
studies are more correctly interpreted as 
giving additional support for methods 
of tuition emphasizing "understanding" 
rather than methods of routine drill. The 
question to be answered, however, i5 
whether "understanding" itself, as meas- 
ured by success in transfer, is enhanced 
by greater or lesser amounts of informa- 
tion given as guidance. 

A distinction must also be made be- 
tween "amount of information" and 
"number of clues." It would be possible 
to design an experiment in which con 
trasted groups receive varying numbers 
of clues. The clues, moreover, might 
differ in the amount of information each 
made available (that is, in the extent to 
which the S was enabled to discriminate 
between possible alternative courses of 
action) (4). Whatever the discrimination 
permitted by the least explicit of the 
clues, this discrimination would also Þe 
made with each additional and more 
direct clue. The effect of such repetition 
would be to reduce errors and increase 
understanding quite apart from the fact 
that with each additional clue, addi 
tional information might be provided. It 
would follow that, where the effectiv® 
ness of differing amounts of informatio? 
is the focus of interest, a necessary an 
perimental control would be that the 17" 
formation any particular § receives. 
the same information at each repetitio? 
of clues, and that the number of repe 
tions be the same for all Ss. 

A number of experiments have been 
reported where the two experiment? 
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safeguards described above were given 
consideration. Typically, in these 
studies, Ss given a general principle, 
underlying a related cluster of prob- 
lems, were less successful than were Ss 
whose attention was simply called to cer- 
tain structural features of the problem 
situations (5, 12, 16, 22, 23). Since the 
application of the generalization should 
lead directly to solutions, these results 
have also been taken as supporting evi- 
dence for the superiority of methods of 
tuition allowing for greater search and 
“discovery.” But an alternative explana- 
tion is possible which would take ac- 
count of the observation that informa- 
tion may differ in kind as well as in 
amount. 

Duncker has suggested that problems 
are usually attacked in stages, the solver 
setting up a series of subproblems (5). 
Protocols of problem-solving experi- 
ments would indicate that, while a few 
individuals may begin by trying to dis- 
cover a generalization which may, when 
found, be applied to all variations of the 
problem, the more typical process is to 
attempt first to discover a method of at- 
tacking and solving specific examples (6, 
18). Generalization may or may not fol- 
low success with a number of problem 
variations. If this is so, then the greater 
effectiveness of information related to 
a method of attacking specific examples 
of a problem-type may be explained as 
arising not from any inherent difference 
in the awareness or activity demanded of 
the S, but from the appropriateness of 
the information to the task the problem 
solver has set himself. Information given 
about the principle involved may be 
quite inappropriate in the early stages of 
problem solving. 

If guidance toward the discovery of 
an attack on a problem (method informa- 


tion) and information about a general 
principle underlying a cluster of similar 
problems (rule information) are, in fact, 
two kinds of information relating to two 
different aspects of problem solving, then 
not one, but two, criteria of success of 
guidance would be indicated; viz., suc- 
cess in solving the problems, and success 
in verbalizing the rule. Where the infor- 
mation made available is appropriate to 
the criterion used, then the guidance 
should prove beneficial. Where the in- 
formation is inappropriate, guidance 
should prove ineffective, if not actually 
detrimental. 

Further, if information about rule and 
about method are two kinds of informa- 
tion, it is possible to envisage guidance 
that would make available to the stu- 
dent greater or lesser amounts of either 
kind of information. If the measure of 
effectiveness is appropriate to the kind 
of information made available, and if 
the less explicit information proves more 
effective, then additional support would 
be given to the position that opportunity 
for greater self-directed search by the 
student is desirable as guidance. If, on 
the other hand, the more explicit infor- 
mation proves more beneficial under 
these conditions then the inference 
would be that greater and more direct 
help is superior. 

To argue that understanding how to 
solve a problem and understanding the 
principle underlying the solution may 
be independent does not imply that both 
understandings may not be acquired and 
integrated. Given a limited period of 
time in which to learn to solve a type of 
problem, the $ with superior intellectual 
ability would be more likely to accom- 
plish such an integration. 

In summary, then, the desirability of 
making available lesser as against greater 
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amounts of information in the guidance 
of problem solving has been a disputed 
question. Many of the experiments 
which have been interpreted as arguing 
for the efficacy of lesser amounts of in- 
formation are more correctly interpreted 
as comparing methods involving prob- 
lem solving as against methods of rou- 
tine drill. There are other experiments, 
however, where all Ss were engaged in 
the discovery of the solution, and where 
lesser amounts of information proved 
more beneficial than did greater 
amounts. One possible interpretation of 
these results is that there is an advantage 
accruing to the 5 when the opportunity 
for search is not restricted. An alterna- 
tive explanation has been suggested 
which would attribute the failure of the 
more explicit information to the inap- 
propriateness of the kind of information 
presented. 


III. AN EXPERIMENT 


Hypotheses 


To test the effectiveness of varying 
amounts and kinds of information as 
guidance in problem solving and with 
due regard for the argument above, а 
number of hypotheses were formulated. 

It was predicted that only information 
appropriate to the criterion employed 
to evaluate success would be effective as 
guidance. Specifically, where the crite- 
rion was the Ss' success in solving prob- 
lems, both instructional and transfer, 
only information about the method of 
attacking specific examples would prove 
effective. Conversely, where the criterion 
was success in verbalizing a principle, 
only information about the principle 
would prove helpful. In addition, it was 
hypothesized that, where the informa- 
tion was inappropriate to the criterion 
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employed, no significant effects would be 
discerned as a result of the guidance. 

It was further hypothesized that 
given an appropriate criterion, the effec- 
tiveness of the guidance would increase 
directly as the amount of information 
supplied the 5 increased. That is, where 
the solution of problems was the crite- 
rion measure, the more information given 
about the method of attacking instruc 
tional examples, the more successful the 
Ss would be. Similarly, where stating the 
rule was the criterion, the more infor- 
mation about the rule which was give? 
as guidance, the more often the principle 
would be correctly generalized. 

Finally, it was hypothesized that the 
eflects of the varying kinds and amount 
of information would be similar for КЫ 
of higher and lower intellectual ability: 


The Instructional Tasks 


To test these hypotheses a problem 


was required which met certain condi- 
tions: (a) the problem had to be 0" 
with which most Ss would not have ha‘ 
prior experience; (b) the problem had 
to be susceptible to numerous variations: 
and (c) all of the variations had to be 
solvable by the application of a commo! 
principle. The "match-task" describes 
by Katona (g) met these three condition 
An example of this problem-type 
shown in Fig. 1. 

The directions which accompany : 
problem shown in Fig. 1 are: “The line? 
in the drawing represent matches. They 
have been arranged so that there are five 
squares in the drawing. Can you таё 
four squares out of the five by moving 10 
more than three matches?” 

There are four restrictions which must 
be observed in solving the problem: (@ 
all of the squares іп the answer must 
of equal size; (b) all of the squares in the 


S. 
is 


the 
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answer must be closed; (c) all of the 
matches must be used in the answer; and 
(d) all of the sides of the squares in the 
answer must be only a single match in 
thickness. 

It will be noted that there are 16 
matches in the example. Since four 
squares are required, it follows that no 
match can serve as the side of more than 


ing by the removal of the least number 
of matches. Knowledge of the principle 
presumably would help the student to 
identify these “critical” squares, but it is 
possible to solve the tasks without be- 
coming aware either of the number of 
matches in the problem, or of the fact 
that solution depends on eliminating 
matches serving a double function. The 


Fic. 1. Sample of Katona match task. 


one square in the answer. Matches must 
be moved, therefore, so that no match 
serves such a double function. This is 
the principle common to all variations 
used in the present experiment. 

'The principle is not a formula in the 
sense that knowledge of it will give the S 
an automatic understanding of which 
matches are to be transposed in any 
given variation of the task. Moreover, 
as Katona points out, solution of the 
task may be approached quite independ- 
ently of knowledge of this rule. Solution 
of the problems may result from per- 
ceiving "holes" in the presentation fig- 
ures, Since the number of matches which 
one is permitted to transpose is limited, 
successful solution will hinge on finding 
those squares which can be completely 
eliminated from the presentation draw- 


rule does serve to limit the alternatives 
before the S, and should give the prob- 
lem solver an important clue as to the 
appearance of a correct answer. 

The task shown in Fig. 1 was used as 
a pretest to screen out Ss with prior 
knowledge of the problem. The Ss were 
also asked to indicate whether they had 
ever seen the problem, or ones like it. 
Those who responded in the affirmative, 
and who solved the trial problem, were 
eliminated from the experiment. 

Three tasks were used as instructional 
problems. ‘The first of these was a simple 
inversion of the puzzle shown in Fig. 1; 
the square E of that drawing being pre- 
sented below, rather than above, square 
D. The second of the instructional tasks 
represented only a slight modification; 
square E of Fig. 1 was shifted to a posi- 
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поп above square В. The last of the 
three instructional tasks was a somewhat 
more difficult drawing; square A of Fig. 
1 was shifted to a position above square 
B so that the five squares of the presenta- 
tion drawing formed a U. 


The Instructional Variations 


The amount of information given as 
instruction to the discovery of the rule 
was varied as follows: 


No information. No information was given 
about the rule, nor was any indication given that 
a common principle existed. 

Some information. The Ss were told that a 
rule existed, were asked to look for this rule, 
and the fact that there were 16 matches in the 
presentation drawing was called to their atten- 
tion. 

Much information. The Ss were told that a 
rule existed, and were told the rule: “Matches 
should be moved so that no square has a com- 
mon side with another square. Every match 
should be the side of only one square.” 


The amount of information given as 
guidance to the discovery of the method 
of solving the examples was also varied 
in three ways: 


No information. No information was given, 

Some information. The Ss were directed to 
shade in certain of the squares, and their at- 
tention was called to the fact that four matches 
remained which were the sides of unshaded 
squares. The clue has the effect of calling at- 
tention to the critical squares. 

Much information. For the three tasks used 
for instruction the Ss were told which matches 
to move where, and were asked to practice this 


solution, 

The clues called “some information” 
were similar, both in the context of a 
search for the rule and for the method, 
in that the Ss’ attention was called to the 
structural relationships inherent in the 
tasks. The clues designated “much infor- 
mation” made available highly explicit 
guidance and reduced the necessity for 
an intensive search of the problem by 
the S. At no point, however, was a com- 
pleted solution of any task shown to any 
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S. Every student had to discover a correct 
response, but this discovery was with the 
aid of more or less direct help from the 
instructions. 

In combining information about rule 
and method, the following pattern was 
adhered to: (а) a statement of the task 
to be solved, (b) information about the 
rule, if any, and (c) information about 
the method, if any. Instructions followed 
this pattern on each of the three prob- 
lems on which guidance was given. 

Combination of information given 
about the rule and about the method of 
solving examples produced nine instruc 
tional variations, as follows: 


1. None on rule, none on method. 
2. None on rule, some on method. 
3. None on rule, much on method. 
4. Some on rule, none on method. 
5. Some on rule, some on method. 
6. Some on rule, much on method. 
7. Much on rule, none on method. 
8. Much on rule, some on method, 
9. Much on rule, much on method. 


Thus, the instructional variations rep 
resented a systematic increase in thé 
amount of information made available 
to the 5 from a treatment in which the 
students were given no information ât 
all to one in which highly directive in 
formation was given about both the rule 
and the method. 

The trial problem, a page illustrating 
and explaining the restrictions, and thé 
three instructional problems (each on ? 
Separate page) were developed into ап 
Instruction Booklet to facilitate grouP 
administration of the experiment (3): 
There were nine variations in the prac 
tice booklets, each representing one 0 
the alternative treatments described 
above. The combination of information 
on rule and method that any particulat 
S received was the same for each of the 
three instructional problems the $ r€ 
ceived. The presentation of each task 
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was accompanied by a space in which 
the 5 could make provisional tries, and 
also an answer column in which he 
could indicate the matches to be trans- 
posed in each problem, and could make 
à drawing of the answer if it had been 
discovered. 


The Test Problems 


Eight additional variations of the 
match task were formulated as test prob- 
lems. The first three of these were simple 
reversals of the tasks used for instruction. 
The final five tasks differed either in 
that triangles меге substituted for 
squares in the presentation drawings, or 
in that the problems were made more 
complex than those used for instruction. 
For example, Test Task 6 gave the S 
seven squares which had to be changed 
to five by moving no more than three 
matches. Test Task 7 presented five 
squares in the form of a Swiss cross; the 
compactness of the drawing seemingly 
creating special difficulties. 

Thus, two measures of transfer were 
provided. These measures will be identi- 
fied as "simple transfer" for the three 
reversal test problems, and "complex 
transfer" for the remaining test tasks. 

Finally, a page was given the S on 
which to write a verbalization of the 
principle common to all variations of 
the match task which he had encount- 
ered. 

The test problems and the final page 
for writing the rule were brought to- 
gether in a Test Booklet, completely 
independent of the various Practice 


Booklets (3). 
The Subjects 


The materials were administered to 
255 twelfth-year students whose median 
age was 17 years, 3 months. Five stu- 
dents were eliminated on the basis of 


prior acquaintance with the tasks. An 
additional 17 students were absent when 
the measure of mental ability was se- 
cured and were not included. The re- 
maining 233 students represented ten 
classes in eight high schools in New York 
and New Jersey suburban communities. 
А wide variation existed in the facilities 
provided by these eight schools. Based on 
information made available by the Met- 
ropolitan School Study Council, Teach- 
ers College, it was determined that ex- 
penditure per pupil ranged from as low 
as $165 to $495. It seems a tenable as- 
sumption that the Ss had had widely 
different experiences and types of guid- 
ance in problem-solving situations. 


Procedures 


The two booklets were administered 
to the Ss as follows: 


Step 1: Introduction (three minutes), The in- 
struction booklets were distributed randomly 
within each class. After the students filled in 
background data, an explanation of the purpose 
of the experiment was read, The Ss were told 
that neither their intelligence nor ability was 
being tested and that the results of the experi- 
ment would not affect their school grades, but 
that the experimenter was seeking information 
on how students learned to solve problems. 

Step 2: Trial problem (one minute). Students 
were asked to attempt the trial problem, At the 
end of one minute, students were asked to write 
“Yes” if they had ever seen the specific task or 
problems like it before; "No" if the problem 
was new to them. 

Step 3: Explanation of restrictions (two min- 
utes). The restrictions were read aloud. Students 
were then asked to study illustrations of the 
improper solutions. At the end of two minutes, 
they were informed that they could look back at 
the page of restrictions during the remainder of 
the practice period. This statement was repeated 
twice during the instruction period. 

Step 4: Instruction period (twelve minutes). 
The students were told that they would have 
twelve minutes, that they could work at their 
own best speed, and that they could divide the 
instruction period between the three tasks as 
they thought best. Finally, they were informed 
that since all of the instruction booklets were 
different, they would not be doing the same 
thing as other students around them. At the end 
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of the twelve minutes all instruction booklets 
were collected. No questions were answered dur- 
ing the instruction period. 

Step 5: Test period (sixteen minutes). After 
the test booklets were distributed, students were 
informed that they would have sixteen minutes 
to solve as many of the cight test problems as 
they could. Again, they were told to work at 
their own best speed, and to divide the test 
period among the problems to suit themselves. 
No questions were answered. 

Step 6: Writing the rule (three minutes). At 
the conclusion of the test period, students were 
told that they would have three minutes to 
formulate a rule. They were instructed that 
they might look back at the test problems they 
had attempted if this would help them formu- 
late a rule. 


With the exception of the trial prob- 
lem, no attempt was made to assure 
equal time being spent on each of the 
tasks. Rather, only the total time as- 
signed to the instruction period, to the 
test, and to the writing of the rule, was 
controlled. Experience in preliminary 
tryouts of the material indicated that 
withdrawing incomplete tasks was dis- 
turbing. Many students abandoned the 
problems after several had been re- 
moved in this fashion. The arbitrary de- 
cision to control the total block of time, 
rather than time spent on each problem, 
while creating some additional difficul- 
ties in the analysis, seemed preferable 
Since a major source of frustration for 
many students was reduced. More im- 
portantly, to have withdrawn the tasks 
would have penalized most those groups 


given the least amount of information 
as instruction. 


Test of Mental Ability 


All Ss were given Form A of the Otis 
Self-Administering Test of Mental Abil- 
ity, Higher Examination (16). The test 
publishers report a reliability of .92, us- 
ing comparable forms. The twenty-min- 
ute score on this test was employed as 
an indicator of the Ss’ mental ability. 


This measure is best thought of as а 
power score, involving both speed and 
level of ability (1). The use of time limits 
during both the instruction and test 
periods argued for the use of this score 
as the control of initial ability rather 
than the untimed Otis score. 

Students scoring 45 or better on the 
Otis were designated as "Highs," Or 
higher than average in mental ability 
(the mean score for all Ss was 45.3) and 
those scoring 44 or lower were called 
"Lows," or lower than average in mental 
ability. 


IV. RESULTS OF THE EXPERIMENT 


Number of Students 


The number of students receivinÉ 
each of the nine variations in instruc 
tions on the match task was not exactly 
equal, as shown in Table 1. However: 
when tested against the marginal totals 
the number of students given each form 
of instruction did not prove to be dis- 
proportionate. For this reason it WA 


TABLE т 


NUMBER or STUDENTS RECEIVING VARYING 
KINDS AND AMOUNTS OF INFORMATION 
AS INSTRUCTION 


= س‎ = — 
Information about | |. eals 
Information the Method Tot 
about the Rule |~—_______——| Ru 
7 
None | Some | Much m 
Much Highs 12 15 тї 3 
Lows 16 то 14 40 
Ѕоте Highs 18 14 13 48 
Lows 8 12 16 3 
None Highs 13 12 14 T 
Lows тї тї 13 m — 
Totals Highs 43 дт 38 127 


А re 
Tests for hypothesis that cell frequencies 5i 
proportional: Highs: x?=2.5; Lows: X7 
X .05, 4 df — 9.5. 
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possible to make comparisons of the ef- 
fectiveness of the varying amounts of 
information with assurance that the 
comparisons would not be unduly af- 
fected by the existence of unequal num- 
bers in each of the treatment groups. 


Mental Ability 


An analysis of variance of the mean 
mental ability scores for the nine treat- 
ment means did not provide a basis for 
rejecting the null hypothesis for either 
the more or less capable students. In 
the comparisons which follow, therefore, 
it was possible to infer that mental abil- 
ity had been equated for the groups re- 
ceiving each of the instructional varia- 
tions. 


Instruction Problems 


The students were given twelve min- 
utes to attempt the three instruction 
problems. Since the time the students 
spent on each of these problems was not 
controlled, the possibility existed that 
the students would attempt different 
numbers of the problems, depending on 
the kind and amount of information 
made available to them as guidance. To 
test this possibility, the mean number 
of instruction problems attempted was 
computed for each of the variations. The 
“number of problems attempted” was 
defined as the number of the last of the 
three problems on which some evidence 
was present that the task had been con- 
sidered by the student (sketch lines, 
doodling, etc.). 

The analysis of the differences among 
these means was accomplished by the 
application of distribution-free tests of 
significance. These tests were employed 
to avoid making untenable assumptions 
both about the normality of the under- 
lying score distributions and the homo- 


geneity of treatment variances. These 
procedures, as adapted, were used 
throughout the remaining analysis, and 
are discussed in Appendix A. 

When these tests were applied, signifi- 
cant differences in the number of instruc- 
tion problems attempted were found to 
exist, attributable to the amount of in- 
formation the student received about the 
method. Among the students of higher 
ability, as the explicitness of the infor- 
mation offered about the method in- 
creased, the number of instruction prob- 
lems attempted also increased. Among 
the students of lower than average abil- 
ity, however, while the students who re- 
ceived "some" or “much” information 
attempted more tasks than did students 
who received no information, the differ- 
ence between those for whom the form 
of help wall me" rather than “much” 
was not reliable. 

No significant differences in the num- 
ber of instruction problems attempted 
could be discerned arising from varia- 
tion in the amount of information the 
students had received about the rule. 
Nor did the fact of having received a 
certain combination of rule and method 
information result in significant differ- 
ences in mean number of instruction 
problems attempted. 

To take account of these differences 
in the number of instruction problems 
attempted, in analyzing success in solv- 
ing the instruction tasks, the percentage 
of those problems attempted which were 
solved correctly was computed for each 
of the treatment groups. These percent- 
ages are given in Table 2. 

The instruction problems (and the 
transfer problems as well) were scored 
as either right or wrong. To be credited 
with having solved the problem, the 
student had to indicate both a correct 
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TABLE 2 


PERCENTAGE OF INSTRUCTION PROBLEMS Ат- 
TEMPTED WHICH WERE SOLVED BY STUDENTS 
RECEIVING VARYING KINDS AND AMOUNTS 
OF INFORMATION AS INSTRUCTION 


Information About 


Information the Method | Totals 
About the Rule Rule 
None | Some | Much 
Much Highs 21 58 go 59 
Lows 13 55 65 44 
Some Highs 23 53 90 | 57 
Lows oo 47 45 40 
None Highs 30 74 86 69 
Lows o9 5o #17 | S 
Totals Highs | 28 | бї | 88 | 6r 
Lows | то 51 62 42 


Differences attributable to kind of information 
received 


2 


Kind of x 
information af 


Highs | Lows 105 р 


Method informa- 


tion 2 18.0 | 14.4 6.0 
Rule information 2 1.7 8.4 б.о 
Method Xrule 

information 4 4:0] 2.8] 9.5 


Differences attributable to amount of 
information received 


U Test 
Comparison Method Rule 
.05 
Highs | Lows | Lows 
Much—no 
information 3.6 3.6 .о 2.0 
Much—some 
information 2.5 Tg 4 2.0 
Some—no 
information 2.5 3.3 |—1.1 2.0 


set of matches to be transposed, and had 
to make a drawing of the required fig- 
ure corresponding to the matches 
marked for transposition. 

"There were instances in which the stu- 
dent showed a satisfactory answer draw- 
ing without marking a correct set of 
matches to be moved, and the opposite 


sometimes occurred. In the majority of 
cases, however, the two parts of the 
answer were either both correct or both 
incorrect. This dependence argued 
against the use of partial scores. The 
use of a simple right or wrong score in- 
troduced less chance error, since it was 
not necessary to determine whether the 
movement of the "critical" matches гер" 
resented an accidental event or indicated 
some partial understanding of the task. 
For students of higher than average 
mental ability, the percentage of in 
struction problems solved increased ne 
the amount of information made avail- 
able to them about the method was ay 
creased. Students given no informatio” 
about the method attempted and solved 
fewer of the tasks than students give? 
some information, and these, in turn, 
attempted and solved fewer problem’ 
than students given the greatest amount 
of information about the method.  . 
For students of less than average abil- 
ity, differences in the percentage of the 
instructional tasks solved also accom 
panied the proffering of informatio” 
about the method of attacking the ex 
amples. For these students, however, "® 
one-to-one correspondence was estab- 
lished between the amount of inform? 
tion presented about the method and thé 
percentage of tasks solved. Giving tPS 
less able students information resulte 
in an increased number of instructi" 
problems attempted and solved, Bus 
making available highly directive infor 
mation did not prove to be substantially 
superior to giving less explicit clues- 
For both the more and the less ab 
students, making available informatio” 
about the rule underlying the tasks 
not result in significant differences in th 
percentage of the problems solved. ^ 
anything, information about the гше "f 
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sulted in fewer problems being correctly 
solved. Among students both above and 
below average in ability, the groups 
given no intimation that a rule existed, 
and not told the rule, solved more of 
the problems than did those who re- 
ceived this information, although the 
differences were not reliable. The detri- 
mental effects of information about the 
rule appeared to be somewhat more 
marked among the less able students. 

No significant effects were found in 
the percentage of instructional tasks 
solved which could be assigned to the 
students having been given a particular 
combination of information about the 
rule and method. 


Simple Transfer Tasks 


The first three test problems were 
simple reversals of the tasks which had 
been used for instruction. All of the 
reversals were attempted by all but seven 
of the students, and these seven were 
distributed among the treatment groups. 
The three problems provided a measure 
of “simple” transfer, since a direct appli- 
cation of the solutions discovered during 
the instructional period was possible. 
The mean number of simple transfer 
problems solved correctly is reported in 
Table 3. 

Differences in the number of simple 
transfer problems solved were found 
which were attributable to the amount 
of information about method which the 
student had received. For both students 
above and below average in ability, the 
effect of having been given some infor- 
mation about the method was to increase 
the number of simple transfer problems 
solved. For the more capable students 
the number of problems correct in- 
creased as the amount of information 
which had been made available in- 


TABLE 3 


MEAN NUMBER OF SIMPLE TRANSFER PROBLEMS 
SOLVED By STUDENTS RECEIVING VARYING 
KINDS AND AMOUNTS OF 


INFORMATION 
Я Information About 
Information the Method Totals 
About the Rule Rule 
None | Some | Much 
Much Highs C5 2.5 2.0 2.0 
Lows 4 1:4 -9 .9 
Ѕоте Highs “9 EO 2.6 1.7 
Lows .о 1.0 a6 
None Highs 1.6 К, 1.9 1:7 
Lows .5 1.3 1.7 1.2 
Totals Highs 1-3 1.9 3 1.8 
Lows 3 1.3 1.0 “9 
Differences attributable to kind of 
information received 
Kind of x 
п : ЁЁ ےک ےا‎ 
information Highs | Lows | .о$ 
Method informa- 
tion 2 8.4 | 14.0 6.0 
Rule information 2 wal sx] 6:9 
MethodXrule | 
information 4 10.72 | 9.4] 9.5 
Differences attributable to amount of 
information received 
U Test 
— € —— 
Comparison Method Rule 
.05 


Highs | Lows | Lows 

Much—no 

information 2.3 2.4 |—1.1 2.0 
Much—some 

information .9 |—1.0 1.2 2.0 
Some—no 

information 1.8 3.4. |—1.8 2.0 
Much 4-some— 


noinformation | 2.2 3.4. |—т.7 2.0 


creased, although only the difference be- 
tween the group of students given the 
most explicit form of information and 
the group given no information was re- 
liable. For the less able students, on the 
other hand, the students who had been 
given the less direct clues solved most of 
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the problems, although the differences 
between the two groups given some form 
of help were not significant. Thus, the 
high ability students were helped most 
by the more direct information, while 
the less able students did not find such 
help more effective. 

Having had varying amounts of infor- 
mation made available about the rule 
did not result in any substantial differ- 
ences in simple transfer. While the dif- 
ferences attributable to the three 
amounts of information given about the 
rule were not in themselves reliable, the 
order of performance is of some interest. 
Among the high ability students, those 
given a statement of the rule solved 
more of the simple transfer problems 
than those not given the rule. Among 
the less able students, those not given the 
rule or a clue to the rule, solved the 
most problems. 

For the students of less than average 
ability there was no significant effect re- 
sulting from having been given a partic- 
ular combination of information about 
both rule and method. Such an effect 
was present in the performance of the 
more able students. To evaluate this in- 
teraction between the two kinds of in- 
formation among the more able stu- 


dents, the percentage of correct solutions 
was determined for each 


group. 

Compared to the group of students 
who had received no information either 
about the rule or the method, the two 
treatments in which the student re- 
ceived an explicit clue in combination 
with a less explicit clue were signifi- 
cantly superior. Thus, being told the 
rule but receiving only a clue to the 
method, or being given directive infor- 
mation about the method with only a 
clue to the rule appeared to be highly 


treatment 


effective forms of guidance for the more 
able students. None of the other com- 
binations of about both 
rule and method resulted in a perform- 


information 


ance on the simple transfer problems 
substantially different from the treat- 
ment in which the student received no 
guidance at all. 

The first three test problems were very 
directly modeled on the problems used 
for instruction, and it has been shown 
that there were significant differences in 
the number of instruction problems at 
tempted and solved. The question arose, 
therefore, as to what the effects of the 
treatments would have been on success 
with the simple transfer tasks if all stu- 
dents had solved a similar number of 
instruction. problems. 

Since success on both the instruction 
and simple transfer problems was ak 
fected by the treatments, to adjust the 
transfer scores for the instruction prob- 
lem scores would have resulted in гето“ 
ing part of the treatment effects. But if 
an adjustment were made, and there still 
remained significant differences attribut 
able to the treatments, then the inter 
pretation would be that not all of the 
differences on the transfer problems are 
explainable in terms of the number © 
instruction problems attempted al“ 
solved (2, p. 82f). The inference woul 
be that there were effects arising {ro™ 
the variations in the amount and kin 
of information quite apart from the per 
centage of problems solved during the 
instruction period. я 

The adjustment was appropriate only 
for the students of higher than average 
ability, since only for this group cou 
the hypothesis of no linear regression be 
tween the two measures be rejected. 

No reliable differences in the mea? 
of the treatment groups remained once 
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„Ше simple transfer scores had been ad- 
justed for percentage of instruction 
problems solved. It may be inferred that, 
for the students of greater ability at 
least, there were no effects resulting from 
having been given varying kinds and 
amounts of information as guidance, 
apart from those which «could be ex- 
plained by the fact that differences ex- 
isted in the number of instruction prob- 
lems attempted and solved. 


Complex Transfer Tasks 


The final five problems of the test de- 
parted in form, though the principle 
underlying the solution remained the 
same, from the tasks which had been 
used for instruction. These problems 
permitted a second measure of transfer. 
Since these problems were less obvious 
variations of the instruction tasks, pre- 
sumably the students whose initial ex- 
perience with the match tasks had en- 
tailed the greatest search should have 
had some advantage on these problems. 

By withdrawing the final task from 
the analysis, differences in the number 
of complex transfer problems attempted 
by the students in the various treatment 
groups were removed. On the four re- 
maining tasks no significant differences 
were found in the number of problems 
attempted. (Using the subgroups de- 
scribed in Appendix A as replications, 
chi squares of 13.8 for "Highs" and 
10.6 for "Lows" were obtained with 
critical value for chi square .o5 equal to 
15.5.) Means for the treatment groups 
on these four problems are given in 
Table 4. 

For the more capable students neither 
the variations in the amount of infor- 
mation about the rule nor about the 
method produced significant differences 
in the number of complex transfer tasks 


TABLE 4 


MEAN NuMBER OF COMPLEX TRANSFER PROB- 
LEMS SOLVED BY STUDENTS RECEIVING 
VARYING KINDS AND AMOUNTS OF 
INFORMATION AS INSTRUCTION 


Information About 


Information the Method Totals 
About the Rule |———— ———— ——— | Rule 
None | Some | Much 
Much Highs Ts 2.8 2.2 ar 
Lows E 1.0 7 8 
Some Highs fX 1.3 tF i 
Lows E Tek 4 “7 
Хопе Highs 2:9.] д4 I$ 1.8 
Lows m li? Tod 1.2 
Totals Highs 1.5 2.0 2.1 1.9 
Method Lows 6 1:3 .8 .9 


Differences attributable to kinds of 
information received 


Kind of df x 


iS ERG سد‎ 
information Miris lows] ves 


Method informa- 


tion 2 6.9 | iiy] б.о 
Rule information 2 Lat 4.4 | 6.0 
Method X rule 

information 4 10.7 | 10.7 9.5 


Differences attributable to amount of 
method information received 


U Test 


Comparison - 
Highs | Lows | .os 


Much—no information 5.5 .6 2.0 
Мисһ-ѕоте . 
information —0.5 |—- 1.6 2.0 
Some—no information 1.6 2.3 o 
Much 4-some—no 
information Li 1.6 2.0 
solved. Students given information 


about the method did solve more prob- 
lems than students given no such in- 
formation, but the differenees in per- 
formance were not reliable. 

Among the less able students, how- 
ever, the students who had had the less 
explicit form of information about the 
method were successful in solving more 
of these tasks than were students given 
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no information about the method. The 
students given highly directive clues to 
the method did not differ significantly 
in performance on these problems from 
students given less explicit aid or those 
given no information at all. 

Again, no substantial differences in 
performance on the complex transfer 
problems resulted from the student hav- 
ing been given varying amounts of in- 
formation about the rule. 

Among both the students of more and 
less than average ability the groups 
given no instruction about the method 
did better relatively, on the complex 
transfer problems than they had on 
either the simple transfer problems or on 
the instructional tasks. As a result, for 
these four complex problems it was not 
possible to conclude that the students 
given information about the method, 
either in the more or less explicit form, 
were superior to those given no infor- 
mation of this kind. 

For both ability-classes of students, 
differences were found on the complex 
tasks resulting from the combination of 
information received as guidance about 
the rule and method. 

Using the students who had received 
no information about either rule or 
method as a comparison group, two com- 
binations of instruction were found to 
differ significantly among the more ca- 
pable students. "These were the group of 
students who had been given only a clue 
to the rule, and the group given only di- 
rective information about the method. 
Both of these groups solved substantially 
fewer of the complex problems that did 
the students given no help at all. Solving 
the greatest number of the tasks were the 
groups given direct information about 
the rule together with less explicit in- 
formation about the method and vice 
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versa, but these students were not so 
superior to those given no information 
at all that chance could be excluded as 
an explanation of the difference. 

On the other hand, among the stu- 
dents of less than average ability, the 
groups given only a clue to the method, 
either in the direct or the less explicit 
form, and given no information about 
the rule, were superior. None of the 
other groups of students did significantly 
differently than the group given no in- 
formation at all. 

Thus, there was little relationship be- 
tween the order of treatments among the 
high and low ability students. The stu- 
dents of lesser ability profited most when 
guidance consisted of information about 
the method alone, and when no mention 
was made of the rule. The better stu- 
dents, however, found these forms of 
help less effective, and were benefited by 
having received information about the 
rule in addition to information about 
the method. 

Success on the simple and complex 
transfer problems was related and the 
hypothesis of no linear relationship 
could be rejected. The complex problem 
scores were adjusted for their regression 
on the simple transfer scores and the 
resulting means were compared. No 
reliable differences in the number of 
complex problems solved by the various 
treatment groups remained once this 
adjustment had been made. Since the 
adjustment removed part of the treat 
ment effects, the failure to find differ- 
ences may best be interpreted as indicat 
ing that no effects, beyond those ех 
plained by the differences in the number 
of simple transfer problems solved, were 
present. Thus, success on the more diffi- 
cult transfer problems was explained bY 
prior success on the simpler tasks. And, 
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as has been shown Гог the better students 
at least, success on the simpler transfer 
tasks was associated with the students 
having solved a greater number of the 
instructional problems. 


Writing the Rule 

As their final students were 
allowed three minutes in which to write 
a statement of the principle involved in 
the problems they had attempted. They 
were permitted to look back over their 
work on the match tasks to help formu- 
late this rule. 

A statement of the rule was taken to 
be correct if it referred to the fact that 
the double function of matches in ‘the 
presentation drawings had to be elimi- 
nated. This principle was sometimes ех- 
pressed in terms of the statement used 
in the directions, “No match should be 
the side of more than one square." But 
any formulation implying “separateness 
was accepted. Thus, “The squares 1n the 
answer have to be spread out" was also 
counted as correct. Between these two 
degrees of ргесіѕепеѕ in verbalizing the 
rule many variations of expression were 
possible, ' | 

Only 21 per cent of the 233 students in 
the experiment were successful in gue 
rectly verbalizing the rule by this cr 
terion, And since only 12 of the students 
of lower-than-average ability succeeded 
in so doing, it seemed inappropriate > 
analyze the results separately for t н 
two classes of students. The percentage 0 
all students in each of the nine treat 
ment groups correctly stating the rule is 
given in Table 5. 

When the measure of 
the information given 2$ p 
ing the ا‎ period was cipi 
to verbalize the principle, only the 
amount of information given about the 


task, 


effectiveness of 
guidance dur- 
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TABLE 5 
PERC) 3E OF STUDENTS CORRECTLY 
V ALIZING PRINCIPLE OF MATCH 
Tasks 
; Information About 
Information the Method Totals 
About the Rule |— Rule 
None | Some | Much 
Much 39 28 2 33 
Some II 1I 28 17 
None 29 04 o7 13 
Totals Method 27 15 22 21 
Differences attributable to Кіпа of 
information received 
Kind of information df х? | x? .05 
Method information 2 3.7 | б.о 
Rule information 2 12.4 б.о 
Method Xrule 
information 4 6.7 | 9.8 
Differences attributable to amount of 
information about the rule 
Comparison U U.os 
Much—no information 2.5 2.0 
Much—some information 6 2:0 
Some—no information І 2.0 


rule resulted in significant differences. 

Neither the information which had 
been given as guidance to the method, 
nor the rule and method combinations 
produced reliably different levels of 
achievement. 

The students who had had the most 
specific statement of the rule more often 
stated the rule correctly than those given 
a clue to the rule or given no informa- 
tion about the rule at all. Those given 
a clue to the rule were able to verbalize 
it no more often than those given no 
information at all. 

While the amount of information 
given about the method did not make 
for significant differences in successful 
statements of the rule, the order of the 
three amounts of such information is 
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of some interest. Those students who re- 
ceive no information about the method 
were most often able to state the rule, 
followed by those for whom highly direc- 
tive method information had been the 
form of guidance. The students given 
only a clue to the method stated the rule 
least often. This order parallels that for 
information about the rule, where the 
criterion was success in solving the prob- 
lems, for all students on the instruction 
problems, and for the "lows" on the 
simple and complex transfer problems. 

A correct statement of the rule was 
not dependent on the student's prior 
success in solving the problems. For the 
total stüdent group (both "lows" and 
"highs" combined) the following correla- 
tion coefficients were obtained: 


Success on rule and instruction problems 


Success on rule and simple transfer prob- 
lems ..... 

Success on 
problems 


rule and complex transfer 


None of these correlations was signifi- 
cantly different from zero. (Correlations 
were between corresponding measures 
for total subgroups as described in Ap- 
pendix A. The critical value of the co- 
efficient equals .4.) 


М. INTERPRETATION ОЕ RESULTS 


QA number of experiments have been reported 
in which seemingly less explicit clues proved more 
effective as guidance in problem solving than 
did more direct information, as, for example, in- 
formation about the principle basic to solution 
The superiority of the less explicit information 
in these situations has been attributed to the 
more active search demanded of the student 
and to the possibility that, аза result of his 
more intensive search, the student would become 
cognizant of the structural relations obtaining t 
the problems. e 

An alternative explanation was Offered which 
suggested that these experimental findings arose 
from a misdirection of the search, It 


E 5 was argued 
that information about principles and Bier 
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tion about a method of attacking specific ex- 
amples of a problem type may be two kinds of 
information, relating to two different subtasks. 
Guidance given to the discovery of the solution of 
one of these subtasks, it was suggested, might be 
inappropriate or even detrimental to the solution 
of the other, at least in the early stages of solving 
a problem. 

It was predicted that where the criterion of 
success was appropriate to the kind of informa- 
tion which had been given as guidance, signifi- 
cant effects attributable to the proffering of that 
information would be discerned, but that where 
the information given and criterion were IM- 
appropriate, no such effects would be found. 
These predictions held for the present data. 
Only where the information given was con- 
sistent with the criterion used to determine 
success were significant differences revealed, On 
the other hand, information not in harmony with 
the criterion used proved no more cilicient than 
no information at all. 

The Ss were apparently presented with two 
tasks rather than one: (a) to discover a method 
of attack on the problems, and (b) to discover a 
satisfactory generalization of the principle. The 
kind of information given served to direct the 
search to one rather than the other of these 
tasks. And, since the more usual pattern in solv- 
ing problems is first to discover a method of at- 
tacking the problem, information given about 
the principle would seem to introduce special 
difficulties for the problem solver. In any сас, 
an active search may be assumed to have 0С 
curred whatever the kind of information made 
available to the student as guidance. The dire 
tion and focus of the student's search may have 
varied, however, depending on the kind of iP 
formation he was given. 

It was further hypothesized that there would 
be no significant difference in the effects of th 
two kinds of information for students of nigh 
and low intellectual ability. This null hypothesis 
was not substantiated. In the solution of both 
the instruction. and transfer problems (simple 
and complex), the detrimental effects of having 
received an inappropriate kind of information 
appeared somewhat more marked for student? 
of less than average ability. On the other hand; 
оп the transfer problems, the differences. at 
tributable to having received an appropriate 
kind of information were more marked for thes? 
less able students. Apparently, for students о 
lesser ability, the power of information to direc 
or misdirect was greater than it was for the more 
able students. 

_ This difference in the power of the inform" 
Чоп proffered to direct or misdirect may be €* 
plained as reflecting a greater ability of the more 
able students to integrate the two kinds of ine 
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formation and apply it to the solution of the 
problems. While no significant interaction was 
discerned either in the solution of the instruc- 
tion problems or in the verbalization of the rule, 
such effects were present on the transfer prob- 
lems. On the simple transfer problems, the stu- 
dents of greatest mental ability who had received 
a combination of information about the rule 
and method were superior, and on the complex 
transfer problems this superiority was main- 
tained, although on these problems the differ- 
ences found were not reliable. On the other 
hand, for the less able students, interaction was 
found significant only on the more complex 
tasks, and with these students it was precisely 
those who had received only information about 
the method, and no information about the rule, 
who were superior. -— 

To suggest that information about principles 
and about methods of attacking a problem are 
two kinds of information does not imply that 
a combination of the two kinds of information 
cannot be understood and applied in problem 
solving. To derive a principle from an under- 
standing of method, or to derive a method of 
attacking problems from an understanding of 
principle, and to integrate and apply the two 
understandings, would require more time than 
to discover either method or principle alone. 
Where the time allowed for problem solving is 
limited, students of highest mental ability might 
be expected to accomplish this integration morc 
often than students of lesser ability. Given 
enough time, relatively easier examples, and 
greater success in solving the easier tasks, stu- 
dents of less-than-average ability might also be 
expected to benefit from receiving both kinds of 
information. But apparently these conditions did 
not obtain for the poorer students in their work 
with the match tasks. 

It may be concluded that while information 
about principle and about method are two kinds 
of information, the effects of having received 
one rather than the other kind are more likely 
to be noted in the early stages of learning to 
solve a class of problems, and more likely to ap- 
pear among poorer than among more able stu- 
dents. (And, though the experiment does not 
provide for this, it might be hypothesized that 
the effects are morc likely to appear as the diffi- 
culty of the problem to be solved is increased.) 

If information about method and about rule 
are accepted as two kinds of information, the test 
of the effectiveness of greater as against lesser 
amounts of information as guidance must be 
made where the criterion used to measure effec- 
tiveness is appropriate to the kind of informa- 
tion supplied. It was predicted that where an 
appropriate criterion was employed, the effec- 
tiveness of guidance would increase directly as 


the amount of information supplied the S in- 
creased. It was hypothesized that this relation- 
ship would hold both for students of greater and 
lesser mental ability. 

Where the criterion was success in solving in- 
structional and transfer problems, the predicted 
relationship between the amount of method in- 
formation received as guidance and success was 
more characteristic of the students of greater 
ability, though it was not fully substantiated 
even among these students. On the instruction 
problems, it was true that the more method in- 
formation the able students were supplied, the 
greater the percentage of the problems attempted 
and solved. This direct one-to-one relationship 
began to weaken on the simple transfer prob- 
lems. On these tasks students who received in- 
formation on the method were superior to those 
who had received no information, but the differ- 
ence between the students who had received the 
greatest amount and those who had Ъсеп given 
less explicit clues was not reliable. On the more 
complex transfer tasks, no reliable differences 
could be noted between the Ss receiving the 
three different amounts of information. Thus, 
while there appeared to be an initial advantage 
accruing to those receiving the greater amount 
of method information, as the more able stu- 
dents tried more of the problems, this initial 
advantage did not persist. 

For the less able students advantages resulting 
from having been given highly directive infor- 
mation about method were even less marked. For 
these students, not only did the predicted re- 
lationship between amount of information about 
method and success in solving examples fail to 
appear, but, in fact, some evidence was secured 
to demonstrate that the less explicit form of 
guidance was preferable. On the complex trans- 
fer tasks, for example, only the Ss receiving the 
less explicit form of information were superior 
to those receiving no information at all. And 
since the more able students receiving the great- 
est amount of information as instruction failed 
to maintain their initial advantage, there is some 
support for the advocacy of those methods of 
guidance which require greater search and which 


lead to greater awareness of structural rel 
ships. 


But this interp. 
factory. In the fi 
"structural" 


ation- 


retation is not entirely satis- 
rst place, the "awareness" or 
h explanation does not make it clear 
why, if less explicit clues were beneficial for 
the less able students, such methods were not 
even more beneficial for students of greater than 
average ability. Second, if having given the stu- 
dents “some” information proved relatively more 
advantageous on the transfer problems than 


“much” information, and if this is explained by 


the greater search required, then why did not 
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the students receiving no information about the 
method do best of all? If a little search is a good 
thing, should not more search have been even a 
better thing? Third, why did those given the 
lesser amounts of information about the rule do 
less well in writing the rule? If search is helpful 
in and of itself, then similar benefits should 
have been exhibited on this task as well. 

An alternative explanation of the results ob- 
tained in this study is possible. On the transfer 
tasks the principal differences were accounted 
for by the number of instruction problems at- 
tempted and solved correctly. For the complex 
tasks no reliable differences remained, attribut- 
able to the treatment variations, once the differ- 
ences explained by prior performance on the 
simpler tasks had been taken into account. On 
the simple transfer tasks, in turn, no differences 
remained once variation in the percentage of in- 
struction problems solved had been removed. 

The advantage of having been given some 
guidance to the method was that it enabled the 
student to attempt and solve more of the initial 
problems in the limited time allowed. Whether 
the more or the less explicit form of guidance 
proved more effective depended upon the ability 
of the student to understand and apply the in- 
formation given. The extent to which informa- 
tion was understood and utilized was not solely 
a function of the amount given. The better stu- 
dents were able to apply directly to the task at 
hand greater amounts of information. Less able 
students were less likely to comprehend the im- 
plications involved in the greater amounts of in- 
formation, could less often apply it, and for 
these students, even on the initial problems, no 
differences appeared as between those given 
direct information and those given only a 
structural clue. 


The fact that the more able students could 
make use of the more direct form of information 
may also explain the results obtained in stat- 
ing the rule. On this task, students for whom the 
rule was identified succeeded more often than 
those given a clue to the rule, and these suc- 
cessful students were, with but a few exceptions, 
all drawn from the upper ability group. 

Moreover, the differences in interaction for the 
two groups of students are also explained by the 
differences in their ability to make use of infor- 
mation given. Among the better students it was 
those given a combination of rule and method 
information who did best, but among the poorer 
students it was precisely the groups given in- 
formation about the method, and none about 
the rule, who did best on the transfer tasks, 

As the students moved from the instruction 
problems to the simple transfer tasks and then 
to the more complex tasks, the differences re- 
sulting from their having had a certain type of 
guidance rather than another 


became less 


marked. This was true both for the more and 
less able students, but more especially tor those 
of greatest ability. This suggests that the bene- 
ficial effects of guidance lie in the speed with 
which the student is brought to his best level 
of performance. Understanding, to the level of 
the S's ability, is likely to be achieved. irrespec- 
tive of the kind and amount of guidance offered. 
If the guidance is appropriate, and if the stu- 
dent is able to make use of the greater amounts 
of information, he will be helped to attain his 
maximum level most quickly by being given 
more direct forms of aid. But if the direct in- 
structions cannot be applied by the student, less 
direct forms of instruction may prove just AS 
beneficial. 

The present study has dealt with the solu- 
tion of intellectual problems. On the basis of 
the results obtained it might be argued, but 
by analogy only, that in the guidance of emo- 
tional problems the efficacy of nondirective 
guidance may result from the inability of the 
$ to apply specifically directive information tO 
what are, for the Ss, extremely difficult prob- 
lems. 

The ability to verbalize a principle without, 
at the same time, a corresponding ability t? 
apply the principle to the solution of problems, 
is not generally accepted as a satisfactory OUt 
come of guidance. With this in mind, the re 
sults of the present experiment carry certain 
implications for guidance in situations similar 
to those experimented with herc. 

The learning situation used was characterized 
by the following conditions: 1, students were 
engaged in intellectual problem solving—they 
had to discover the solution for themselves: 2 
the problem was one in which previously ace 
quired generalizations could not be readily ар” 
plied; 3, the principle underlying the problem 
was not a formula in the sense that it Pt 
scribed specific steps needed to solve any given 
variation of the problem; and 4, the time given 
to instruction and problem solving was "° 
stricted. a 

In situations meeting these conditions, the = 
sults of the study suggest: 

1. Information given the student about ре 
method of solving examples is more likely to x 
beneficial than information given about on 
principle—at least in the initial stages of pt? 
lem solving. " 

2. Some appropriate guidance will pO" 
more helpful to the student than no guidance 
Leaving the student to discover for himself thé 
solution of a problem will not prevent under 
Standing, but will probably delay it. 

3. The effectiveness of guidance does not дЕ" 
pend solely on the amount of information im 
parted. More explicit forms of instruction we 
prove most helpful with students able to арр? 


the 
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the information. For students of lesser ability, 
less explicit clues, designed to highlight struc- 
tural relationships, may prove just as effective. 


VI. SUMMARY 


An unresolved issue in the guidance of 
problem solving is the desirability of 
making available greater rather than 
lesser amounts of information. Informa- 
tion may be of two kinds; it may relate 
either to the principle or to method. It 
was hypothesized that performance 
would improve as the amount of infor- 
mation given as guidance about method 
of solution or principle for solution was 
increased. 

Several forms of the match task were 
presented to twelfth grade students. In 
guidance there were three variations in 
the amount of verbal information given 
about the principle: "No information": 
"Some information," a clue to the princi- 
ple; and "Much information," a state- 
ment of the principle. Similarly, students 
were given "No," "Some," and "Much" 
information about the method of solving 
examples. The combination of the 
amounts of information about method 
and principle yielded nine different in- 
struction treatments ranging from one 
with no information about either the 
principle or the method, to one with 
highly directive information about both 
principle and method. Success was ap- 
praised in the number of instructional 
tasks solved, in transfer to simple and 
complex problems, and in verbalizing 
the principle separately for students 
above and below average in mental 
ability. 

For students above average in mental 
ability, success in solving problems in- 
creased only for the instruction problems 
directly as the amount of information 
given about method increased. For stu- 
dents of lower ability, some appropriate 
guidance was superior to no guidance. In 


solving the problems both for students 
above and below average in mental 
ability, information about the rule did 
not seem to affect results differentially. 

For the simple transfer problems, the 
more capable students seemed to profit 
most from the combination of explicit 
information about the rule with an in- 
direct clue to the method, or from a clue 
to the rule in combination with explicit 
information about method. On the com- 
plex transfer tasks, the high ability stu- 
dents receiving only information about 
the method, or only an indirect clue to 
the rule, did less well than students given 
other combinations of information. But 
on these complex tasks, students of less 
than average ability given only guidance 
to the method, in either the direct or less 
explicit form, were superior. 

No treatment effects remained for the 
complex transfer problems once an ad- 
justment had been made for differences 
on the simple transfer tasks. Similarly, 
for the high ability group, at least, no 
treatment effects remained on the simple 
transfer problems once scores were ad- 
justed for differences in success on the 
instruction problems. 

Success in writing the rule increased 
as the amount of information students 
had been given about the rule increased, 
although only the students given the 
most explicit information about the rule 
were superior to those given no informa- 
tion about the rule. In writing the rule, 
no effects were noticed which could be 
attributed to the amount of information 
the students had received about the 
method of solving examples. Success in 
verbalizing the rule was uncorrelated 
with success in solving the problems. 

These results were interpreted to in- 
dicate (a) that information used in guid- 
ance must be appropriate to the task 
set for the student, (b) that some appro- 


priate guidance is beneficial, but that 
failure to provide it will delay rather 
than prevent solution, (c) that the effec- 
tiveness of guidance does not depend 
solely on the amount of information im- 


parted, but that (d) more explicit in- 
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struction will prove most helpful with 
the more able students while (е) less 
explicit instruction may be just as effec- 
tive as more directive guidance for the 
less able students. 


APPENDIX А 


STATISTICAL PROCEDURES 


‘The analyses of variance reported in the text, 
with the exception of the analysis of mental 
ability means, were carried through on ranked 
measures, In this way, the assumptions of nor- 
mality of score distributions and of homogeneity 
of treatment variances were avoided—assump- 
tions which in most cases were untenable, It is 
interesting to note that the practical difference 
of this use of distribution-free statistics was 
not great. As a check, all of the analyses were 
carried through in the more traditional manner 
in disregard of the assumptions. Except for a 
few instances in which the nonparametric test 
statistic proved to be very close to its critical 
value, the decisions reached were identical under 
either procedure. However, the distribution-free 
tests described below were the techniques of 
choice for the present data. 

For each of the nine treatments, and for 
both the "Highs" and "Lows," three subgroups 
were formed. This was done by assigning stu- 
dents to the subgroups, in order, from a list 
of the ranked mental ability scores for each 
treatment. For example, for the treatment: "No 
information on rule; no information оп 
method," the subgrouping was as follows: 


Otis Score 


69 
бо 
54 
54 
54 
53 


46 


Subgroup 


— ا فن ت ою‏ ون 


Students in the other treatment groups were 
assigned to subgroups in a like manner. The 
composition of the subgroups remained con- 
stant for all of the analyses, of course. 

The next step was to compute means (or, in 
some cases, percentages) for each of the sub- 
groups on the particular measure being studied. 
To test the main effects of the three variations 
in information about the method, the variations 
in information about the rule were treated as 


replications, with comparable subgroups ranked. 
To test the main effects of the variations 1n 
information received about the rule, the process 
was reversed, and the comparable subgroups on 
method were treated as replications. The tes 
itself consisted in determining the probability 
of the obtained rank totals for, first, the three 
variations in method information, and then for 
the three variations in rule information. . d 
formula used and the test is more fully describe! 
by Moses (15) and Freidman (7). Я 

To test Ше significance of the interaction: 
Rule x Method, the procedure suggested. y 
Wilcoxon (24) was employed. ‘The test statist 
was computed as a sum of two values. One 
component was obtained by tabulating the di 
ferences between corresponding subgroup values 
for the "Much" and "Some" rule variations, от 
each of three variations in method information: 
The second component was obtained by finding 
the difference between the sum of the value 
of the "Much" and "Some" rule variations an 
twice the value of the "None" rule variation 
comparable subgroups—again, for each of * é 
three method variations. These differences ке 
then ranked and the test described above E 
applied to the rank totals. The argument А 
the procedure is that, assuming there is ап E 
vantage accruing from having received à pe : 
ticular combination of rule and method informa" 
tion, the differences will be consistently 8762 ne 
(or smaller) for each of the replications © : 
treatments so devised. 

In those cases where the main effects V 
shown to be significant by the procedure test 
scribed above, the Mann-Whitney (11) “y in- 
was used to analyze the difference between ей 
dividual means of the variations. The test i$ V it 
known and does not need to be described, bU m 
needs to be pointed out that the individual CO” 
parisons were made by ranking the eighteen ue 
group means for any two variations under 60 
sideration. xx 

Finally, it should be noted that correlation 
and tests for linearity, which are reporte i, 
the text, were also based on the subgroup en 
mates on the measures of interest. 
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I. THE PROBLEM 


HE IMPORTANCE ОЁ understanding 
Ti relationship between personality 
factors and problem-solving behavior has 
recently become apparent (4, 10, 18). 
The purpose of the present study was to 
investigate several hypotheses concern- 
ing the relationship beween personality 
and problem-solving factors under nega- 
tive conditions. That an individual’s re- 
action to a negative state of affairs in the 
environment is important becomes ap- 
parent when one notes the numerous 
personality constructs employed to con- 
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a research grant M-955 from the National Insti- 
tute of Mental Health of the National Insti- 
tutes of Health, Public Health Service, and in 

* part by the Rockefeller Foundation for re- 
search in perception and social psychology. 
We are greatly indebted to the staff members of 
the following participating organizations and in- 
stitutions for their generous cooperation: YMCA 
camps with headquarters in Trenton and 
Newark, N.J.; Ten Mile River Boy Scout Camps 
(N.Y.); Sequassen Boy Scout Camp (Conn); 
Valley Regional High School, Deep River 
(Conn); Hamden High School (Conn); Notre 
Dame High School, West Haven (Conn); Fair- 
field High School (Corin); Perth Amboy High 
School (N.J); Olney High School, Philadelphia 
(Pa); The Peddie School, Hightstown (N.J.); 
and The Hun School, Princeton (N.J.). We 
would also like to express our appreciation to 
Our research assistants Mr. Ronald S. Wilson 
and Mr, Arthur Adlerstein. 
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ceptualize such reactions: stress toler- 
ance, frustration tolerance, ego strength, 
and tolerance for ambiguity. It is im- 
plicitly assumed in the use of such con- 
structs that a person's inability to cope 
with environmental adversity is more 
or less central to many adjustment prob- 
lems; such an assumption is explicitly 
supported in the repeated incidence 
noted in case histories of a negative con- 
dition in the patient's life situation pre- 
cipitating a breakdown in adjustment. 
"Therefore, we felt justified in our selec- 
tion of failure and criticism conditions 
for study, and have accordingly employed 
these situational conditions not only as 
experimentally induced conditions, but 
also as conditional items in personality 
measures, Such a restriction of condi- 
tional (or situational) variance represents 
our agreement with Rotter (19) on the 
importance of taking account of, and if 
possible controlling, the psychological 
situation. It is not unlikely that much of 
the previous difficulty in accounting for 
interindividual variance in problem solv- 
ing by the use of personality instruments 


? A part of one section (V) also noted reactions 
to praise conditions, but here also the relevant 
personality items were appropriate to the ex- 
perimental condition in question, i.e., prais 


— 
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(e.g., 8) has been due to the implicit lack 
of recognition of such situational speci- 
ficity, as evidenced by the previous use 


response, along with three other meas- 
ures. 
Section III describes а personality 
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of general measures such as the Ror- 
schach. 

This study investigates a number of 
hypotheses bearing either directly or in- 
directly on the defensive process of 
avoidance. Other descriptive terms such 
as denial or blame-projection (17) are 
closely related to our use of avoidance. 
In our use of the term avoidance we 
will mean a behavior or interpretation 
directed away from a situational event. 
For example, in the situation of failure, 
a person’s quitting a task when he is 
performing poorly may be considered 
failure-avoidance behavior, while а per- 
son’s refusal to perceive the possibility of 
his own inadequacy under these same 
situational circumstances may be re- 
garded as an interpretation directed at 
avoiding failure. 

Section II describes a method for meas- 
uring four behavioral variables in a 
single problem-solving situation under 
conditions of failure and presents the re- 
lationships between these measures. 
Using primarily speed and error meas- 
ures, previous studies (e.g, 13) have 
demonstrated that failure increases inter- 
individual performance variation. Re- 
garding this disruptive effect of failure 
upon the performance of certain subjects 
(Ss), Lazarus and Eriksen state, “It is 
also possible that some Ss did much 
worse on Test 2 because they became 
disgusted by the whole situation, or so 
threatened that their response was to 
give up or to “psychologically leave the 
field’ (13, p. 103, italics ours). Since 
avoidance is a potentially important re- 
sponse to failure, we therefore devised a 
behavioral method to permit an objec- 
tive measure of this "leaving the field" 


measure intended to assess the prob- 
ability of a person's making an avoidant | 
interpretation to a failure situation, 
i.e., refusal to perceive that he might be 
failing. In Section IV the relationship 
between this failure-avoidant interpre 
tation measure and failure-avoidant be 
havior is explored. Implicit in our com 
ception of failure-avoidant interpretation 

is an underlying defensive process which 
prevents the individual from lowering 
his evaluation of himself under negative 
conditions. Sections V and VI therefore 
investigate the relationship betwee? 
failure-avoidant interpretation and sell- 
evaluation following experimentally in 
duced negative conditions. 


II. RELATIONSHIP OF FAILURE-AVOIDA 
BEHAVIOR TO EXPECTANCY, ВАМ ОКА! 
ALTERATION, AND ERRORS IN А 
PROBLEM-SOLVING SITUATION 


s " s to 
This section has two aims: (4) 


develop a method for describing an indi- 
vidual’s behavior in a problem-solvin& 
situation along four behavioral йр, 
sions—expectancy, looking for alternati" 
solutions, quality of performance, p? 
failure-avoidance; and (b) to study t r 
relationship of failure-avoidant behav”, 
to the other three measures. These V?! à 
ables are referred to as behavioral men 
ures because each one can be meast? 
by direct observation. Although the р js 
ent study used behavioral measures n 
assumed that such data differ from ре 
sonality test data only іп the шеше, 
employed and not in the nature of p 
variables involved. After measuring Ё [ 
interindividual variability on several 

havioral variables, one can then und". 


hen niit 
take the problem of accounting for | 
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variance through the use of personality 
constructs and derived instruments. 

The importance of failure-avoidant 
behavior has been previously discussed 
and may be re-emphasized by noting that 
if an individual leaves or avoids the 
problem it is obvious that there can be 
no solution. Therefore, because avoid- 
ance occupies this central position in the 
problem-solving process, all hypotheses 
were formulated relative to avoidance. 
The other three variables selected for 
study have been employed and discussed 
previously in the literature, though not 
always in a problem-solving context and 
not always with the same operations 
used here. Definitions of the four meas- 
ures follow: 

a. Failure-avoidant behavior: extent to 
which § avoids a situation in which he is 
receiving gradually increasing failure by 
selecting a different goal rather than con- 
tinuing to strive toward the same goal. 

b. Expectancy or goal-getting behavior 
(20): the score which S reports that he 
actually expects to make on the next 
trial, i.e., the traditional level-of-aspira- 
tion measure. 

c. Behavioral alternation or looking 
for alternative solutions (23): the num- 
ber of times S changes his solution o: 
approach while striving toward the same 
goal. This variable has been previously 
assumed to be of importance as indicated 
by the following statement from Eriksen, 
Lazarus, and Strange: "Giving subjects 
false information that they are failing 
may cause some of them to alter their 
mode of attack on the problem" (8, p. 
284). 

d. Quality of performance (6): relative 
€rror measure based on pre- and post- 
failure performance. 

The following hypotheses were tested: 

1. That failure-avoidant behavior will 


be directly related to expectancy, i.e., Ss 
who set high goals will be more likely to 
avoid the failure situation. Since high 
goal-setting behavior has been previously 
regarded as “unrealistic” (18, 21), we as- 
sumed that such behavior therefore rep- 
resented one form of avoidance. 

2. That failure-avoidant behavior will 
be inversely related to behavioral alter- 
nation, ie., Ss who use fewer solutions 
will be more likely to avoid the failure 
situation. In a previous study investigat- 
ing behavioral alternation, Schroder and 
Rotter (23) have suggested that rigid be- 
havior (i.c, not looking for alternative 
solutions) may be viewed as one form of 
avoidance as follows: '"The avoidance 
behavior itself may be regarded as a 
single solution which is regularly rein- 
forced by preventing the reoccurrence 
of some trauma, thus providing the con- 
ditions making for single solution learn- 
ing" (23, p. 148). 

3. That failure-avoidant behavior will 
be directly related to performance dis- 
ruption, і.е., Ss who make more errors 
after failure will be more likely to avoid 
the failure situation. This hypothesis is 
based on the assumption that the tend- 
ency to make errors following failure 
stems from the same kind of defensive 
process which leads to failure-avoidant 
behavior. 

In the method described the problem- 
solving situation consists of an insoluble 
sorting problem with six available but 
incorrect solution pathways to a desired 
goal. The 5 is free to use as many or as 
few of the available solutions as he 
chooses, and is also free to avoid the 
situation by choosing an alternative 
goal (i.e., a different problem), or to con- 
tinue to try to reach the same goal. 
Progress toward this goal was indicated 
by controlled scores given by E as well 
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as S’s knowledge that his solution was in- 
correct. In previous studies, failure has 
frequently been induced either by a 
combination of referents, e.g., social 

- reproof accompanied by a low score (22), 
or by very strong failure stimulation (14). 
The present method utilizes a technique 
of minimal and gradually increasing 
failure with increasing trials. An attempt 
was made to specify the failure stimula- 
tion as much as possible so that only the 
score given to 5 would be the referent for 
failure. Since the induction of failure 
irevitably involved an interpersonal sit- 
uation between Е and $, it became im- 
portant to insure that the experimental 
condition and the obtained results could 
be replicated. Therefore, two separate in- 
vestigations using the same method were 
carried out in different localities for pur- 
poses of cross validation. 


A. Method 
1. Subjects 


Group А: 57 male freshmen 
dents at University A. 

Group B: 78 male freshmen who had not been 
in a psychology course, secured from the student 
employment service at University B. 


psychology stu- 


2. Materials 


а. А sorting box containing four slots and so 
constructed that $ was unable to see a card 
after it had been sorted. 

b. Task 1 was an insoluble card-sorting prob- 
lem with six theoretically possible solutions, On 
each trial, a pack of 12 cards was used, each 


card containing a cue variation of six m 


ajor 
concepts: color of card 


lo (blue, yellow, pink, and 
green); position of letter (upright, left, right, 


and inverted); nonsense syllable (MUV E 

NUS, and JOH); form (circle, Sis ee 
and pentagon); number of forms (from one to 
four; and color of form (black, gray, white 
and red) Each major concept was varied 
throughout the 12 cards by the four cue varia. 
tions indicated parenthetically so that one dioc 
in the problem-solving box could be used for 
each variation in the major concept (solution) 


which 5 chose to use. For example, the first 
card was constructed as follows: blue card (card 
color); letter inverted; MUV (nonsense syllable); 
two (number) white (color of form) squares 
(form). The distribution of four cucs within 
each major concept was always 4:3:3:2 SO that d 
was impossible to achieve the instructed goal o 
obtaining an equal distribution without make 
an error. Four different packs were employee 
and the cards were presented to 5 in a constant 
order which maximized the distribution of cues 
over the four slots during the early sorts. i 

c. Task 2 was present only so that E cou " 
point to it during the instructions, since те 
periment was completed whenever 5 left Tas! Е 
Task 2 consisted of a series of questions typed о 
sheets of paper and placed in a folder. nade 

d. Sets of norms which E used ostensibly 
calculate S's score after cach trial. 

€. Score slips and record sheets, used à 
scribed in procedure. 


s de- 


3. Instructions 
, two Es 
1 іп 
pa 


The experiment was carried out by | 
in two different localities using identica 
structions and procedure, The instructions xk. 
sented the problem as follows: Task A pil- 
structured as a measure of decision-making а аз 
ity and leadership potential, and Task 2 jif- 
structured as a different task measuring, А e 
ferent ability, creative intelligence. The m 
tions were designed to induce cgo involve 
and to emphasize the importance of sagot д 
һу pointing out the relationship between р 
measures and postcollege success. Then P 
tinued as follows: pin 

"Task 1 (pointing) measures the manner 
which you handle decision making OF 
generally, your leadership potential. It c s 
of sorting or organizing packs of 12 cars a 
four groups differing along a single ERO 
istic. The aim is to sort the cards into were 
groups (pointing to four boxes)." xamples ot 
then given to insure that S understood the 
ture of the problem. Then E continued: ect ® 
_“When I tell you to start, you will a ut 
single card, look at it, make your decision “rote 
grouping and place it in one of the boxe 5, js 
that you cannot see a card again once ting) 
sorted into one of these sorting boxes (poin po* 
Then select a second card from the supP P , 
and sort it. Continue selecting cards One fou! 
time and sorting them into one of ше дш 


7 n pds 
relevant Sorting boxes until the 12 C! 
used. 


n- 


—— 
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etailed instructions for all procedu 
be obtained from the authors. 
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“Your score will then be worked out by me 
on the basis of the following: 

“First, your score will depend upon the dis- 
tribution of the cards. The more even the dis- 
tribution of the cards over the four boxes, the 
higher the score. Of course, the best solution is 
four Broups of three each. The nearer this is 
approached, the higher the score and the lowest 
score would be for 12 cards in one box. 

“Second, your score will depend upon the time 
taken, so you should work as fast as you can. 
hird, your score will depend upon the 
errors made. An error consists of placing a card 
in the wrong box. 

"And, last, your score will depend also upon 
your general approach to the problem. 

"Unlike most other tests you have probably 
laken, in this task you will be given a good deal 
of information. This is because it is a test of 
your decision making. After each trial your 
Score will be worked out by me. Once the test 
Starts, there should be as little talking as pos- 
sible. In order to facilitate this, I will pass the 
score you actually made on that trial to you on 
а slip of paper like this (hand score slip to 5). 
At this point you can enter your score on the 
record sheet provided (hand record sheet to S) 
for your own information. After you have en- 
tered your score you have two things to record 
on this slip. 

"First, select the t which you wish to try 
оп the next trial. This choice is up to you. It 
is perfectly OK if you want to leave Task 1 and 
try something else after any trial, 

"Second, enter the score that you actually 
expect to make on the next trial. When this is 
done, pass the sheet back to me and wait for 


me to give you the signal to start (take slip from 
5) 


“Now on this task do not pay any attention 
to the back of the cards. The symbols there are 
for identification purposes only and have noth- 
ing to do with the task. Select only one card at 
a time, turn it over so that you can see the 
front, make sure it is not upside down, then 
place it in one of the boxes and select another 
card. Remember to sort on the basis of a single 
characteristic. When you have sorted all 12 
cards, say “finished” so that the exact time 
can be recorded. The materials may differ from 
trial to trial, but the problem remains the 
same. This task is not essentially difficult, but 
your initial reaction to the task will very likely 
be that it is a little complex. However, as you 
go along you will most probably see the basis 
of the solution. Since your score is based ona 
number of different factors—distribution, speed, 
errors, and approach—your score should get 
higher since improvement may occur on any one 


of these factors. That is, you should improve 
as you go along . . . up to a level of somewhere 
between 30 and 4o which is the average for 
college freshmen. Are there any questions? Do 
as well as you can . . . ready begin." 


4. Procedure 


After each trial, E entered the time taken and 
the code number of the cards placed in each 
sorting box. The E then referred to the "norms" 
and entered 55 predetermined score on a slip 
of paper which he passed to $ (in order to 
minimize any possible biasing effects of extrane- 
ous social factors), The slip contained three 
statements: 

“Your score on trial 

Check a. or b. 

a. Desire to continue with task 1 
b. Desire to try taak2 ans 
Write down the score you actually expect 
to make on the next trial " 
The S entered his score on the record sheet 
provided, checked one of the two tasks, recorded 
his expectancy, and returned the slip to E. The 
E inspected the slip to note the task choice, 
placed the next pack of cards in position (four 
packs of cards were rotated throughout the 
trials) and signalled $ to begin the next trial. 
This procedure was continued until S selected 
Task 2 or until the twelfth trial had been 
reached. In cither case, a half-hour interview 
related to S’s attitude toward the problem and 
attempted solutions was then conducted. Follow- 
ing this interview, S was informed of the true 
nature of the experiment and requested not to 
discuss the experiment with other students. 


5. Score Pattern 


The score pattern for consecutive trials was: 
9, 13, 16, 20, 21, 22, 22, 21, 20, 18, 17, and 15 (eg. 
score on the fifth trial was 21). The early scores 
were low relative to the instructions and the 
expectations of college students. The scores ad- 
vanced to a point still below the minimal goal, 
leveled off, and then decreased markedly, It was 
assumed that $ would react to each Score in 
terms of certain elements in the instructions 
which indicated that (a) S should improve with 


practice, and (b) a good score would be go or 
more, 


6. Operations and Scoring 


The cards were code-numbered so that E 
could identify both the concept (solution) used 
on a given trial and the number of errors. 
Objective scoring was possible in all except three 
or four cases where the solution for a single 
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trial was doubtful. In these cases, S's report in 
the interview of the solution used was accepted. 
An error was defined as a misplaced card in 
terms of the concept being used for sorting on 
that trial. Expectancy and avoidance measures 
were directly available on the score slip filled 
out by S. - 
Since S was free to leave Task 1 at any time 
after the first trial, it was necessary to make 
some arbitrary decision regarding the trial 
an S must reach before he would be included 
in analysis. Trial 5 was selected because this was 
the first point at which the score pattern began 
to level off. That is, if S left Task 1 before 
Trial 5 it is likely that he cither misunderstood 
the instructions or placed a higher value on 
Task 2, and in either instance he should be ex- 
cluded. Therefore the use of Trial 5 as the cut- 
off point should provide a good sampling of S's 
behavior on the variables under consideration. 
Of the total of 135 Ss in both groups, 44 were 
dropped for the following reasons: 25 because 
of their selecting Task 2 before Trial 5; 13 be- 
cause of their previous participation in ап 
earlier experiment involving false scores which 
led to their being suspicious of scores as indicated 
in interviews; three because of their color blind- 
ness; апа three because of their misunderstand- 
ing of the instructions and sorting on the basis 
of the code numbers on the back of the cards. 
The following measures were used for cach 
variable: (а) Failure-avoidant behavior was 
measured by trial at which S selected "Task 2 
or left Task 1; since the experiment was dis- 
continued after Trial 12, an avoidance score of 
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(b) Expectancy was calculated. by adding 55 
expectancy for the first four trials. (с) 
Behavioral alternation was measured by count- 
ing the number of solution alternations during 
the first five trials, giving a range of О to t 
(d) Quality of performance. was measured by 
comparing the errors on the same solution 
before and after Trial 5; i.c. as failure stimula- 
tion increased, did errors increase, remain the 
same, decrease? This relative measure was 
caleulated by comparing the number аг er 
S made on a given solution оп Trial б we 
the number of errors made on the previous Dp 
in which that solution was employed. If 5 ик 
a new solution on Trial 6, the n trial is 
which an carlier solution was ava able d 
comparison was used (this procedure of EOS 
ing relative error-making tendencies only на 
the same solution was intended to control Ts 
the differential difficulty level of the six 59 
tions). 


scores 


or 


В. Results 
Table 1 gives the cutting points 
v E pora cid 
number of 5s for each of the four ei 
Ы) J ~ * co 
ables in Group A, Group B, and КҮ 
bined group. With the exception о à 
" „Ж H 1 
error measure, a pooled distributio! 
used 
woul 


and 
wi 


the two groups combined wis 
establish cutting points which 


Y sols 
yield the most nearly equal number M 
Ј procedu e 


13 indicated that 5 did not leave Task 1 at in cach subgroup. Using this sp i 
all (and this would be considered low avoidance). Avoidance and alternating were 
TABLE 1 
GROUP AND COMBINED DISTRIBUTIONS FOR BEHAVIORAL MEASURES 2 
Expectancy ornati 2 
Group —— — А Alternating = 
61-72 73-79 80-89 90-102 o-1 | 2 3-4 ES 
Е а 2 
8 | її | 12 TI | 8 15 it | 16 3o 
А ra 13 | 12 12 | 21 n 17 ee 
Tot P d е ———— cement І 
Гоха! _% 2$ | 23 | " 36 R | E | oh > 
- P di = — = = 
س‎ = „Emors Avoidance 
Decr | f > | : ч m кара: Е 
E ame | Incr. N | 02-13 9-11 | 
A T -— -- == 
В 13 = | = у 4 18 з | ï 
— х | 15 MN. | 16 14 | 19 
Total 24 | 32 26 UNE s аа ہے‎ 
82 | | 7 о 
E = 34 | 3# y “oh 
" e group tota s lor errors are 4o and 42 fı + — ince el 
measures do not include Ss who switched s Ten саша and B rather than 42 and 49 5! 
5 al s. 
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TABLE 2 
RELATIONSHIP OF EXPECTANCY, ALTERNATIN 
Hypothesis Group df pb | & | F | df p 
ЕЕС ERS D „аж А | 
1. Expectancy-avoidance А 6 «.oi | | | 
i B 6 <.10 | | | 
Total | 12 | <.o1 | 41 | 2.02 | 18/72 | «.os 
2. Alternating-avoidance | A 4 <.10 | | 
| B | 4 <.20 
| Total | 15.190 |. 8 | <.06 | .33 3.95 | 4/86 | <.от 
3. Errors-avoidance | هز‎ 14.820 4 <.01 | | 
B | 6.663 | 4 «.20 
| Total 8 <.o1 | 


* The epsilon reported is that of the r 


gression of avoidance on the other variable, e.g., expectancy, 


since the hypotheses were stated in this form and the measures obtained in that order. 


in thirds, and expectancy in fourths. The 
error measure was split according to the 
logical procedure of errors decreasing, re- 
maining the same, or increasing as fail- 
ure stimulation increased since these 
three categories yielded groups of fairly 
equal size. 

The groupings in Table 1 were used 
to calculate chi-square values for testing 
the hypotheses, as summarized in Table 
2. In addition, the degree of relation- 
ship existing in Hypotheses 1 and 2 was 
evaluated by the use of epsilon (since the 
avoidance distribution was obviously 
nonnormal) computed on the basis of 
raw scores rather than grouped data. 
Epsilon was not calculated for Hy- 
pothesis 3 since quality of performance 
was a relative measure. 

Hypothesis т. Avoidant behavior is 
directly related to expectancy. That is, 
Ss who set high goals are more likely to 
avoid the situation while low-goal-setting 
Ss are more likely to continue to strive 
toward the same goal under failure con- 
ditions. 

Hypothesis 2. Avoidant behavior is in- 
versely related to behavioral alternation. 
Although the relationship as reflected 


by chi-square value is at borderline sig- 
nificance (< .06), the epsilon value (p < 
.01) indicates a significant degree of re- 
lationship. This rigidity-avoidance rela- 
tionship was further substantiated by 
observing that failure to alternate solu- 
tion on Trial 6 was related (< ло) to 
avoidant behavior. 

Hypothesis 3. Avoidant behavior is 
directly related to performance disrup- 
tion after failure stimulation increased. 
Since this hypothesis was necessarily 
tested without any reference to these Ss 
for whom the relationship would be 
most appropriate (i.e., the nine Ss who 
left Task 1 at the earliest sign of failure 
on Trial 5 and therefore had no post- 
failure performance measure for com- 
parison), the performance disruption- 
avoidance relationship seems amply sup- 
ported. 

No relationship was noted between 
any two of the other three variables (ex- 
pectancy, alternating, and errors) with 
p levels ranging from .30 to Бо. 


C. Postexperimental Interviews 


The usual function of interviewing Ss 
following conditional experimentation is 
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to obtain a relatively independent index 
of the extent to which the experimental 
variable has been induced. Viewed 
simply, E would therefore be primarily 
interested in S's stating verbally that he 
felt he had failed. However, such a view- 
point overlooks the defensive nature of 
verbalization. Particularly in the present 
study utilizing minimal failure stimula- 
tion and focusing on avoidant behavior, 
we did not expect all Ss to verbalize feel- 
ings of failure. If one considers the inter- 
view protocols from a defensive point of 
view there is no reason to question the 
successful induction of failure stimula- 
tion. However, it is equally clear that 
there is wide variation in the verbaliza- 
tions made by Ss when describing their 
reactions to the situation. 

Since the results of the objective meas- 
ures indicated a configural behavioral 
pattern revolving around failure-avoid- 
ant behavior, we attempted to extend 
this pattern further by noting the verbali- 
zations made by avoidant and nonavoid- 
ant Ss. It is true, of course, that the non- 
avoidant S, by virtue of his staying with 
the task, experienced more intensive 
failure than the avoidant $, but even with 
this difference in experience it appeared 
potentially useful to note his interview 
responses. One of the most fruitful ques- 
tions was "How did you happen to se- 
lect Task 2 at that point?" i.e., in effect, 
"Why did you elect to avoid the situa- 
tion at that point?" When responses of 
Ss who left the task at Trial 5 or 6 (at 
the earliest signs of failure) were com- 
pared with responses of Ss who did not 
leave the task until Trial 12, pronounced 
differences were noted between the 
groups. The verbalizations of the avoid- 
ant group were primarily of the follow- 
ing variety: "I had done as well as I 
could," "Didn't think I could do any 
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better," or "Getting tired of this one." 
On the other hand the responses of the 
nonavoidant Ss were primarily of the 
following nature: "Wasn't doing very 
well so decided I'd better quit," "About 
time for me to give up," and "Better 
stop because I was getting worse." 

In view of these responses it seemed 
reasonable to hypothesize that such varia- 
tion in interpretations of failure might 
partially account. for the configuration 
of relationships which emerged. That 15 
the nonavoidant 5 who persisted in look- 
ing for other solutions with a minimum 
of disruption in performance may have 
behaved in this fashion at least partially 
because he interpreted the situation as 
one in which he was failing. Conversely, 
the avoidant 5 might be described as 
defending himself against failure by be 
having in such a way as to prevent the 
occurrence of any situation which he 
might have to interpret as failure. Fur- 
ther exploration of the possibility of such 
a relationship will be described їп the 
next two sections. 


ПІ. SITUATIONAL INTERPRETATION 
EXPERIMENT 


The situational interpretation exper” 
ment (SIE) described here represent 
method for measuring how an individu“ 
interprets (or perceives) a situation. Th 
primary aim was to measure an ind 
vidual's tendency to make avoidant inte" 
pretations to negative conditions [e 
cism or failure); the secondary aim 8 
to measure an individual's interpretati?! 
of several other situational conditio 
e.g., praise and success. Postexperiment 
data in the first study as well as gene 
agreement in personality theories I°- 
cate that a person's perception or inte Г 
pretation of a situation is of major 
portance in determining which subclass 
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of (the otherwise almost infinite number 
of) behaviors he will employ in that 
situation. Since the intervening process 
of interpretation occupies such a pivotal 
position in determining responses, it 
seemed important to attempt to measure 
such interpretations directly. 

It is presently assumed that individuals 
categorize or summarize their past ex- 
periences into systems or concepts or rel- 
ative constancies (5, 12, 19). Since few 
Situations are ever exactly repeated the 
individual must generalize in order to 
adjust effectively to changing conditions 
Or to utilize past experiences effectively. 
This process of generalization is assumed 
to result in a tendency to behave in a 
more similar manner in two or more 
Situations. 

Central to the present position is the 
assumption that, in order to adjust effec- 
tively to a situation of failure, an indi- 
vidual must admit that he is doing 
poorly, that he is in some way inade- 
quate, or that he is, in fact, failing. We 
assumed that when an S interprets a fail- 
ure situation by thinking '"This means 
I'm not very good at this," that such an 
interpretation implies an admission of 
some personal inadequacy or self-nega- 
tion. It should be emphasized that we are 
not using the term "inadequacy" in its 
usual sense which implies behavioral in- 
effectiveness. In contrast, we mean by 
"inadequate" that the individual is will- 
ing to consider possible weaknesses to 
admit that he may be wrong, thus open- 
ing the possibility for modifying his 
behavior. In considering other possibili- 
ties of interpreting a failure situation it is 
obvious, though nonetheless important, 
that individuals may avoid making 
negative interpretations by a host of 
defensive interpretations, €g; S may 
think “This is too difficult for anyone 


to do. .. . I'm doing all right however," 
or "I was just unlucky." Indeed, the 
entire gamut of defenses emanating from 
avoidance processes may be considered 
as functioning to prevent interpretations 
of personal inadequacy. We will refer to 
this tendency to avoid making self- 
negative interpretations to negative con- 
ditions as avoidant interpretation. 

The method outlined here is referred 
to as a situational interpretation experi- 
ment because the individual is con- 
fronted with a number of hypothetical 
situations and then forced to make a 
choice between two possible interpreta- 
tions of each situation. His choice of 
interpretation then becomes the opera- 
tion for describing the concept the in- 
dividual used for integrating that par- 
ticular situational event. When it has 
been established that an event, e.g., fail- 
ure, falls within a given system of inte- 
gration, e.g., self-evaluation, it is said 
to be interpreted in a given way. 'This 
method allows a number of observations 
of each event, in addition to sampling a 
range of events. It also permits provid- 
ing various potential interpretations to 
be used, although in the present form of 
the SIE we have dealt only with the 
conceptual system of self-affirmation and 
negation. (In considering interpretations 
of positive situations we would not, 
of course, make the same assumption of 
avoidance-adequacy equivalence that we 
made in relation to negative situations, 
since there is no necessary defensiveness 
implied in a person's feeling adequate 
after success.) 


A. Method 


т. Instructions 


All Ss were requested to read the face sheet 
containing instructions which read as follows: 
“This is a test of how well you can make deci- 
sions. The decisions you have to make are not 
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always easy. In each of the questions you arc 
given a short story about something that is 
happening. The story gives all the information 
about what is happening but it does not say 
anything about the thoughts a person would 
have if he were in the story. Your task is to 
complete the story by deciding which thought 
would follow. In cach case one of the two 
thoughts is a better answer than the other. You 
are to select the one which is the truest, Read 
the stories carefully. Ask if you do not under- 
stand anything. Make your decisions accurately,” 


2. Description of Method 


The total scale consisted of go items, but 
the criticism-failure (CF) subscale consisting of 
ten items below was the primary measure. It will 
be noted that this subscale is made up of four 
criticism items, two verbal failure items, and four 
failure items which may be regarded separately 
for certain purposes, As the SIE was presented 
to $ the same stem, i.e., situation, was sampled 
a number of times with different alternatives 
and randomized throughout the test, However, 
the stem and alternatives are grouped together 
below with their position in the test indicated 
by item number, Self-negative alternatives are 
indicated by asterisks, 


a. Criticism condition: 


You are trying to work out some decisions 
about a plan of yours. You hope that the plan 
will be a good one for you. One of the people 
in charge of the project comes by. He glances 
at your plans and says that you don't seem to 
be doing it right, that your plan doesn't seem 
to be very good. Which is the most correct 
thought to have? 

Item #7 (a) "He must be mistaken." 
(b) "I'm just not up to it."* 
Item #10 (a) "This shows that I'm no good 
here." 
(b) “This means that he cannot un- 
derstand my work.” 
Item #15 (a) “He must be right."* 
(b) “This shows he knows nothing 


about it.” 
Item #18 (a) "He knows I'm not doing very 
well."* 
(b) “It's OK, if there's anything 
wrong it's only an unlucky mis- 
take." 


b. Verbal failure condition: 


You are working on a task and hope to do 
well. Someone looks at your work and says that 


* Copies of the full scale in its complete form 
are available on request from the authors, 
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you are not doing very well here. Which is the 

most correct thought to have? 

Item #2 (a) "He must be wrong.” 3 
(b) "I'm not doing very well here. 

Ttem #29 (a) “I still think it's OR" 


сев: 0 
(b) “He can sce my weaknesses. 


c. Failure condition: 


k 8 H ‚ you 
You have been working on a project and p 
have a strong desire to do well and finish = 
At times you have felt that it was giving you 3 


little trouble апа now it seems that you vill 
fail at it. Which is the most correct thought © 
have? ИР 
Item #9 (a) “It is very seldom that I fai ч 
(b) his shows that I'm not up 
its . М 
Item #13 (a) “This shows that the project mu 
be a very hard one” " 
(b) “This shows that I might P 
be so good."* „йй 
Item #19 (a) “This is too difficult for any 
to do.” Js 
(b) “If I cannot do this it is бай , 
Item #27 (a) "I must have been unlucky НЕ 
(b) “This means I'm not very go 
at th 
d. The remaining twenty items were divided 


as follows: 


Praise: four items 
Success: four items 
Help: four items 
Remote praise:* four items 
Verbal praise: two items 
Peer praise: two items 


3. Scoring 
he иш!" 


seS 
spon*', 
es phe 


It was arbitrarily decided to score u 
ber of "inadequate" or nonavoidant T 
in order to arrive at a CF subscale score: 10 
hypothetical range of scores is from O ferred 
and a low score on this subscale will be uae of 
to às avoidant interpretation, In Le 
the other subscales the "inadequate" inte 
tations were also tallied for scoring. crit 

The present scale is the fourth modific? e 
Apart from modifying items which woul Б 
more equal splits, the major change 19 d з 
modification was to substitute ап adult == of 
agent of criticism (and also praise) in р! ie 
the peer figure which had been used іп Ё 


ye’ 


T 
* From S. S. Tomkins, A model of t€ M £ 
being. Unpublished. Remote praise en о! 
situation containing the subject and at теа int 
other person when the other person is 
praised. 
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forms. The effect of this change is discussed be- 
low. 


4. Procedure 


"The SIE was administered to 500 white boys 
between the ages of 13 and 18 during the sum- 
mer of 1955. Тһе Ss were drawn from YMC? 
camps, Boy Scout camps, public and private 
schools. Approximately half of the Ss were se- 
cured on a volunteer basis and half of 
the Ss were tested in total groups. The testing 
Was carried out in groups ranging in size from 
5 to 8o. No time limit was set but the time 
taken ranged from 15 to 30 minutes. In every 
case the SIE was administered. immediately 
following the Sentence-Completion Method (11) 
and was followed by a short verbal intelligence 
test consisting of 20 vocabulary items (27). 


В. Results 


The distribution of scores on the CF 
subscale is presented in Table 3 along 
with the distribution of scores on the 
praise-success (PS) subscale for compari- 
son. As would be anticipated on an 
"alfirmation-negation" continuum, the 
scores on the CF subscale cluster at the 
high (negation) end of the scale 
while scores on the PS subscale tend to- 
ward the low (affirmation) end of the 
scale. That the CF subscale is internally 
homogeneous is indicated by the correla- 
tion of .34 between the four-item criti- 
cism subscale and the failure subscale. 
The CF subscale also appears to be rela- 
tively independent of the PS subscale 
since the correlation between these two 
subscales is low (r = .08). 

For purposes of future investigation it 
may be noted that the CF subscale cor- 
relates .16 with the remote praise and .o2 
with help (for 500 cases, any correlation 
of .og is statistically significant at the 
.o level and a correlation of .12 is sta- 
tistically significant at the .o1 level.) 

An interesting, but tentative, finding 
during pretesting an earlier form of the 
SIE provides some information on pos- 
sible sources of variance in the criticism 


DARD DEVIA- 


FREQUENCIES, MEANS, AN 
ND PRAISE 


TIONS FOR CRITICISM-FAILURE 
Success SUBSCALES 


Criti | Praise- 


Sci ч sm- | 
коте Failure Success 
° | 2 6 
I 3 27 
2 1$ 70 
3 26 8o 
" | 43 тот 
5 08 73 
о 70 67 
rj 95 32 
8 | 83 10 
9° | 58 12 
то | 31 4 
Mean | 6.48 4.26 
SD | 2.10 | 2.03 


condition. At an earlier stage the criti- 
cism stem consisted of a peer agent rather 
than an adult as in the present form, 
and this earlier data indicated that 
peer criticism elicits fewer self-negative 
choices than the adult-induced criticism. 
Such a difference was not noted in the 
agent of praise, so it may be tentatively 
concluded that interpretations made to 
criticism are to some extent dependent 
on the person administering the criti- 
cism. 

Intelligence. The mean and standard 
deviation of the verbal intelligence scale 
were 9.62 and 2.58 respectively. ‘This in- 
telligence scale correlated .10 with the 
CF subscale («.02) but did not relate to 
any of the other subscales. 


IV. RELATIONSHIP BETWEEN FAILURE- 
AvOIDANT BEHAVIOR AND FAILURE- 
AVOIDANT INTERPRETATION 


The primary aim of this section was to 
sutdy the relationship between avoidant 
interpretation and avoidant behavior in 
a failure situation. The secondary aim 
was to attempt a cross validation of pre- 
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vious behavioral relationships noted (Sec- 
tion II), using a slightly different method 
and a different population. 

The operation for avoidant interpreta- 
tion is S’s score on the CF subscale of the 
SIE, with a low score being referred to as 
avoidant. The operation for avoidant 
behavior is quite similar to that used in 
Section II—the trial at which S avoids 
the problem situation in which he is 
experiencing increasing failure by select- 
ing a different goal. 

Measures of interpretation and be- 
havior were obtained according to the 
method to be described in order to test 
the hypothesis that failure-avoidant in- 
terpretation would be associated with 
failure-avoidant behavior. An attempt 
was also made to obtain other behavioral 
measures (used in Section IT) and. other 
test measures, e.g., intelligence, in order 
to clarify further the nature of avoidant 
behavior. However, the matter of crucial 
importance is the utilization of avoid- 


ance as reflected in interpretation and 
behavior. 


A. Method 
1. Subjects 


The Ss were 66 high school boys between the 


ages of 1g and 18, attending two summer 
schools, 


2. Materials 


a. Task 1 consisted of a multiple-choice type 
of symbol-substitution test. Each trial required 
approximately 15 seconds, 

b. Task 2 was a card-sorting task consisting 
of a sorting box identical to that used in II 
and of three demonstration packs and six task 
packs of eight cards each. In all cases at least 
one concept was relevant for sorting. When a 
concept was relevant, a correct sort always pro- 
duced four groups of two cards each. The 
demonstration packs were used to insure that 
S understood the principles of grouping. The 
first three task packs used on the first three 
trials contained five concepts (letter, object, num- 


ber of dots, form, and color) but only опе 
(letters) was relevant for grouping. For ех- 
ample, in the first pack all eight cards con- 
tained the same object (shoe), the same number 
of dots (three), the same form (triangle), and 
the same color (green), but differed with respect 
to letters (two cards with L, two with M, two 
with N and two with O). On the second pack 
the four nonrelevant concepts were again con- 
stant throughout but different from Pack 1. In 
other words, on the first three. packs there was 
interpack variation, but no intrapack variation 
for all concepts except the relevant concept. 
namely letters. The fourth pack contained опе 
relevant concept (letters), but the other on 
concepts changed on Trial 5 to indicate tO ` 
that a change in the situation. had occurred. 
Simultaneously failing scores were given. Packs 
5 and 6 contained five relevant concepts which 
could be used for sorting and were alternated 
from trial to trial, after ‘Trial 5. 

c. Sets of "norms" ostensibly used to compute 
S's score. , 

d. Score sheets and record slips used to 10- 
form S of his score and make his choice of 
"Task 1 or Task 2 after each trial. 


3. Procedure 


The group tests were administered in ips 
following order: (a) The Sentence-Completion 
Method (11), (b) the Situational Interpretation 
Experiment, (c) the Thorndike-Gallup Verba 
Intelligence scale 27). During the week follow" 
ing the group tests, each 5 was seen individu 
ally. In this experimental procedure 5 was wer 
а set of instructions designed to emphasize m 
importance of succeeding on Tasks ! and 2, 
The problems were presented as measures з 
decision-making ability. So that S would sie 
stand the value of any score obtained, һе bs 
informed in both Tasks 1 and 2 as follows 
"In problems like this you will improve bet 
practice; that is, cach time you do it, you анор y 
get a better or a higher decision makini 
score . . . your score should get higher as J 
go along. You should get up toa score б, ре 
tween 30 and 35 with practice which woul Ê 
a good score for high school boys." The ane 
pattern for Task 1 was б, 14, 20, 31. After ae 
fourth trial on Task 1, Æ said to S, “It 100 y 
as though you have learned to do this es 
Well .. ‚ you are doing very well on this ш 
lem, so let us take a look at Task 2 now (P 
cate). Later you might want to come back i 
this problem if you like.” At this point Е ап, 
5 moved to Task 2, plainly marked at the ons 
posite end of the table. The nature of 
Sorting task was clearly demonstrated to ly 
fore beginning. On the first three trials °" 


be 
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One cue was relevant for sorting, but at Trial 
5. all five cues were relevant and simultaneously 
the scores leveled off and began to decrease. The 
Score pattern for Task 2 was 8, 15, 21, 22, 22, 
21, 19, 18, 17, 15. After each trial $ was handed 
a slip containing his score, a space to check his 
preference for Task 1 or Task 2 on the next 
trial, and his expectancy for the next trial, The 
experiment was continued until 5 selected Task 
! or until the twelfth trial. 

Although the present method was intention- 
ally similar in most respects to that of the 
college study (Section 1I) there were two dif- 
ferences: (a) The cards were so constructed that 
$ could not alternate his solution until the fifth 
trial, a modification designed to obtain a better 
measure of postfailure behavioral alternation 
(in the first study Ss varied in their prefailure 
alternating, thereby making postfailure analy- 
Sis difficult). In practice, however, only à small 
Proportion of Ss alternated after failure, so that 
further revision of the method is indicated in 
ae respect. (b) Each S experienced success on 
the alternate task (Task 1), so that avoidance 
ые Present study indicated not only avoid? 
ж the failure situation imposed їп Task 2, 

at also going back to a previously successful 
Fabler or goal (in the first study some $s 
switched to the alternate goal before Trial 5. 
Apparently because they were curious about the 
nature of the other task; so that it seemed worth 
i o give each S a similar experience with 
тав task). Even with this modification, 
from ae it was necessary to climinate 1g ee 
АГ nalysis because they selected the alternate 

al before the fifth trial. 


B. Results т 
1. Avoidant Behavior and Interpretation 


_ Table 4 indicates a significant rela- 
tionship between avoidant interpreta- 
tion and avoidant behavior. The inter- 
Pretation score in Table 4 is based on 
ы criticism-failure subscale score. When 
а similar table based only on the failure 
subscale score is considered, the relation- 
ШР is also significant (<.01) even 

tough this failure subscale consists of 
only four items, 
кы noted previously tha i. 

may avoid an inadequate inter 

Pretation of a negative event in a variety 
а Ways, and actually the avoidant alter- 
Natives on the criticism-failure subscale 


t an indi- 


TABLE 4 


RELATIONSHIP BETWEEN AVOIDANT INTER- 
PRETATION AND AvorDANT BEHAVIOR 


Avoidant Nonavoidant 
TM Interpretation | Interpretation 
Behavior (1-7 on CF (8-10 оп CF 
subscale) subscale) 
Avoidant 19 6 
(4-7) 
Nonavoidant II 18 
(8-13) 


Corrected x?=6.401; p= <.02. 


present different defenses to avoid ac- 
cepting the negative event. Therefore, 
we made an item analysis of the ten 
criticism-failure subscale items (and later 
on the other 20 items also) to note 
whether the selection of a given alterna- 
tive related to either avoidant or non- 
avoidant behavior. In such an analysis 
the selection of an avoidant alternative 
may provide a good predictor which will 
discriminate between extreme behavioral 
groups, but selection of the opposite 
alternative on the same item need not 
be equally discriminating. 

Seventy per cent or more of the Ss 
who selected the following alternatives 
were behavioral avoidant Ss: 


27. (Failure) "I must have been unlucky here." 
19. (Failure) “This is too difficult for anyone 


to do." | | 
18. (Criticism) "It's OK. If there is anything 


wrong, it’s only an unlucky mistake.” 


Seventy per cent or more of the Ss who 
selected the following alternatives were 
behavioral nonavoidant Ss: 

10. (Criticism) “This shows I'm no good here.” 


11. (Failure) “This shows I might not be so 
good.” 


In addition it was further noted that 
the following two items from the help 
subscale were selected by at least 70 per 


cent of the behavioral nonavoidant Ss: 
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12. (Help) “This shows how bad he thinks 
I am." y 

4. (Help) “It means I probably couldn't do 

it myself.” 

These results seem to lend further, 
more specific, support to our initial hy- 
potheses regarding the processes under- 
lying avoidant behavior. The alternatives 
selected by the nonavoidant $ imply a 
willingness to lower his self-evaluation in 
reaction to failure or criticism while the 
alternatives selected by the avoidant S 
imply defenses against any such lowering 
of self-evaluation. 


2. Other Relationships 


No relationship was noted between 
verbal intelligence scores and avoidant 
behavior. There was not sufficient varia- 
tion on behavioral alternation to repli- 
cate the rigidity-avoidance finding. How- 
ever, the relationship between avoidance 
and expectancy was again noted (<.05) 
which provides another cross validation 


for this finding on a different popula- 
tion. 


V. RELATIONSHIP BETWEEN AVOIDANT 
ERPRETATION AND POSTCRITICISM 
SELF-EVALUATION 

The primary aim of this section was 
to investigate the relationship between 
avoidant interpretation and self-evalua- 
tion following criticism. A secondary 
phase, identical in method to the pri- 


mary phase, concerned interpretation 
and self-evaluation under pr 


tions. In both their verb 


aise condi- 


al comments (11) 
and interpretation alternatives selected 


(IV), behavior avoidant Ss 


apparently 
utilized defenses 


against lowering their 
self-evaluation following the occurrence 
of a negative condition. Therefore, 
order to test this relationshi 
cisely, we employed a 


in 
Р more pre- 
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direct measure of 
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self-evaluation as a dependent variable 
in the present study. Contrasting gu 
representing avoidant and nonavoidan 
Ss on the CF subscale of the SIE, were 
individually placed in a performance 
situation and S's self-evaluation бараг 
following E's criticism on a given (ria 
was measured. It may be noted that e 
hypothesis tested. represents an explici 
effort to validate the criticism-failure 
subscale. | 

It was hypothesized that avoidan А 
would be more likely to hold constant 
or increase their postcriticism evalua 
tions while nonavoidant Ss would De 
more likely to decrease their postcriticis! 
evaluations. 

We have stated earlier that we WOU и 
not make the assumption of gu 
avoidance equivalence with respect i i 
praise or success conditions. Indeed, a 
one maintained a relative dennnon 
avoidant interpretation, i.c., S'S impli a 
rejection of the event, we might qu 
that an inadequate interpretation к 
praise would be avoidant. In light of а á 
consideration, we applied the same ai 
proach outlined above to the poe 
of "praise-avoidant" interpretations t! 
postpraise self-evaluation for prelimi? 
exploratory purposes. 


t Ss 


ld 


А. Method 
1. Subjects 


A total of 61 Ss were selected as follows: 


pe 

Куз. s wel 
Criticism condition. Two groups of Mailure 
selected on the basis of their criticism-f2 ad 


subscale score, Eight Ss scoring between о сир 
2 were considered the extreme avoidant 8 on- 
and 18 Ss scoring between 8 and 10 were 

sidered the nonavoidant group. 


А one we 
Praise condition. Two groups of А gub” 
selected on the basis of their praise-succes and 
scale score. Seventeen Ss scoring between 18 


E à and 

10 were considered the avoidant group A d 
Ss Scoring between o and 2 were conside! 

be the nonavoidant group. 
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2. Materials 


A set of symbol-substitution tasks similar to 
the Rotter-Jensen group level-of-aspiration tests 
(20) was used. Each task consisted of a key 
containing 20 symbols represented by 20 alpha- 
betical letters, On cach trial $ had to select and 
enter the correct letter from the key for each 
of 15 symbols, The mean time taken to conr 
plete cach trial was about 45 seconds. For each 
of six trials the symbols were changed, but the 
problems were approximately equal in dilli- 
culty, 

After each trial, § made an evaluation of how 
well he thought he was doing. This was done 
by means of a modified scalometer technique, 
оп a sheet of paper containing 12 squares in 
a horizontal row. The 1st, 4th, 9th, and 12th 
squares were labeled with the words: “Very 
poor," "Not too well,” “Е well," and "Very 
good,” respectively. The words, "How well do 
you think you are doing " were printed at the 
top of the sheet. Just before 5 started another 
trial he indicated how well he expected to do 
on the coming trial. For this purpose the same 
rating sheets were used except that the words, 
‘How well do you expect to do next time?” were 
printed on the top of the page. 


= 


3. Procedure 


The usual safeguards were taken to assure 
that the Es were not aware of the group 10 
Which a subject belonged. In the instructions 
the coding problem was demonstrated to 5 and 
he was informed that his score was dependent 
Upon his speed and accuracy. In this experiment 
S was allowed to finish each problem and Е 
recorded the time. Absolutely no indication of 
the quality of the subjects performance was 
Siven by E at any stage except through the oC 
currence of the praise or criticism. which was 
Biven after Trial g but before 5 was handed 
the evaluation scalometer. In all six trials were 
Biven. After cach trial $ evaluated his past per- 
formance and entered his expectancy for the 
next trial using materials described above. After 
Trial 3 E held S's last performance sheet and 
made one of the following statements: 

Criticism condition. "Well, now - 3 
Not doing so good here. - - it doesn't look like 
You're doing this so well.” I 

Praise condition. “Well, now - - - you're doing 
Okay here , . , it looks like you're doing very 
well," 

Emphasis through intonation was controlled 
carefully through practice апа specific instruc: 
tions to the Es. In both conditions, the critic 1 
adverb in the first half of the statement ("okay 


. you're 


and "not so good") was emphasized by elevation 
of the vocal pitch. In the second half of cach 
statement (after the pause), emphasis on the ad- 
verb (“very well" and "not so well" was 
avoided by maintaining a monotone pitch. 
Avoidance of finality of statement was achieved 
through minimizing the pitch drop toward the 
end of the sentence. 


4. Subjects Discarded 


In this investigation it was essential to insure 
that any changes in evaluation could be at- 
tributed to the experimental conditions. But a 
major extraneous factor is introduced if the 
time 5 takes on cach trial is sufficiently variable 
to become a basis for evaluation. The task was 
selected so that this variability would be reduced 
to a minimum while maintaining some interest 
value, but pretesting indicated that a few Ss’ 
performance times were significantly variable to 
influence evaluation, Consequently, it was de- 
cided to discard all Ss for whom evaluation and 
performance times were significantly correlated, 
Employing a .o5 level of significance as the cut- 
off point, this criterion eliminated 10 Ss from 
the initial sample of 61, 5 from each condition, 
thus leaving 21 in the criticism condition and 
go in the praise condition. 


В. Results 


1. Criticism Condition 


Table 5 indicates that the hypothesis 
regarding a relationship between avoid- 
ant interpretation and postcriticism self- 
evaluation is strongly supported. The 
CF subscale was noted to have a similar, 


though nonsignificant, relationship to 


expectancy. 


TABLE s 


RELATIONSHIP BETWEEN AVOIDANT INTER- 
PRETATION AND POSTCRITICISM 
-EVALUATION 


Avoidant Nonavoidant 
Interpretation | Interpretation 


Evaluation (o-2 on CF (8-10 on CF 
subscale) subscale) 
Increased or 
same 6 E 
Decreased o 14 


Corrected х= 12.863; p= «.oor. 
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2. Praise Condition 


Analysis of the praise condition in- 
dicated that “praise avoidant” interpreta- 
tions were significantly (<.05) related to 
failure to increase postpraise evaluation. 
No relationship was found between ex- 
pectancy and evaluation in either of the 
subscales to absolute evaluation, a meas- 
ure which will be considered in the next 

section. 


VI. RELATIONSHIP Or AVOIDANT INTER- 
PRETATION TO OVEREVALUATION OF 
PERFORMANCE AND ANXIETY 


The study described in the previous 
section was designed to minimize all 
feedback to § except E's comment, i.e., 
S received no “knowledge of results” or 
score on trials preceding E’s comment. 
The present section was initially aimed 
to study the role of ау 
tion as a determinant 
criticism administered 
given score pattern. 


oidant interpreta- 
ОЁ S's reaction to 
in relation to a 
More specifically, 


presented to 5 by E's inte 
cism (negative evaluation) in relation to 
an increasing score pattern (positive ac. 
complishment). We reasoned that S's 
implicit choice between $ 
modes as indicated 
self-evaluation shoul 
tion tendencies, 
ing” the positive 
negative evaluatio. 
nonavoidant Ss, 


rjecting criti- 


"accept- 
Score rather than the 
n, and conversely for 


lation to these scores 
ately rejected. The Score pattern finally 
adopted was more ambiguous than Posi- 


tive, i.e., 5 experienced slight improve- 
ment. 

An inspection of data using this score 
pattern indicated an unanticipated find- 
ing: namely, that the effect of the inter- 
pretation variable was of major impor- 
tance on the initial self-evaluation fol- 
lowing the first (rather low) score (see 
Table 6). This interpretation-produced 
difference was in evidence throughout 
the six trials and therefore masked any 
possible effects of E's evaluative com- 
ment. Therefore, we will present only 
the method and results based on the first 
three (pre-evaluation) trials since the ex: 
perimental induction had a negligible 
effect. 

This study will describe (a) the rela- 
tionship of avoidant interpretation > 
both absolute evaluation and tendency 
10 overevaluate performance, and (0) 
relationship between avoidant interpreta- 
tion and anxiety as measured by the 
Taylor anxiety scale (24). | 

Аз the Study was carried out, Ss wen 
assigned to two conditions since praise 
Was also employed in addition to GNU 
cism. However, since neither evaluation 
was effective and the pre-evaluative sore 
patterns (for criticism—11, 12, 14 and fo! 
praise—15, 


А roup 
15; 14) did not produce gro P 
differences 


in either self-evaluation т 
performance during the first three tria с, 
these conditions were combined for pt! 
Poses of present analysis. 


A. Method 
1. Subjects 


5 
The Ss were 80 unselected high school be 
who were attending a parochial summer sc han 
for purposes of raising their grades, rather t 
ecause of failing course work. 


ә. Materials 
the 
“substitution problems from 


s 20) 
group level-of-aspiration test oe 
€ of symbols was removed 80 
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each problem contained 25 symbols to be com- 
pleted, Evaluation and expectancy after each 
trial were recorded using the same materials de- 
Scribed in Section V. 


3. Procedure 


The Ss were divided into two groups of 42 
and 38 for purposes of varying group testing 
Order. In the first group the testing order was: 
Sentence Completion Method (11); Taylor Anx- 
tety Scale (24); Situational Interpretation Ex- 
Periment; and a short verbal intelligence test 
(27). In the second group the order was identical 
except that the order of SIE and anxiety scale 
was reversed, Following the group testing, Ss 
were assigned to one of four subgroups to 
Counterbalance for two conditions and two Es 
by matching on interpretation variable. 

Experimental procedure was similar to that 
described in Section V except that 5 was told 
he would have a given amount of time in which 
to do cach trial. This procedure permitted E 
to stop each S at the same predetermined num- 
ber of symbols for a given trial. Thus, 5 be- 
came aware of his score without the necessity 
of E's announcing it, since the symbols were 
Plainly numbered in consecutive order. The 
maximum hypothetical score was 25. The pre- 
determined score patterns for the first three 
trials in both conditions were rather low (11, 12, 
Ub and 13, 15, 14) so that in effect both pat- 
terns тау be considered as representing low, 
Possibly failing, scores. 


4- Measures Employed in Analysis 


as Personality measures. 


1. CF subscale score 
2. Anxiety scale score 


+ Behavioral measures. 


1. Absolute evaluation: actual box checked 
following a given trial (possible range 
from o to 12) 

2. Corrected rate: number of correct sym- 
bols per minute. This was used to com- 
pute— 

3. Over- and underevaluation: In order to 
Obtain a measure of evaluation relative to 
performance which would permit identi- 
fication of Ss who overevaluated or under- 
evaluated their performance, the follow- 
ing procedure was employed: ei 
Cutting points were established for 0 
the corrected rate and total evaluation 
Scores for the first three trials. Using these 
two measures, Ss were appropriately 
placed into one of the four resulting 
quadrants, Therefore, Ss in the quadrant 
of high evaluation and low rate of per- 


formance were referred to as “overevalu- 
ators,” and Ss in the quadrant of low 
evaluation and high rate of performance 
were referred to as “underevaluators” 
(Ss in both conditions were grouped to- 
gether, since these measures are based on 
trials preceding E's evaluative comment). 


B. Results 

1. Relationships Between Personality 
Measures 

Pearsonian correlation coefficients com- 
puted between anxiety and criticism- 
failure subscale score yielded a correla- 
tion of —.29, which is statistically sig- 
nificant at less than the .o1 level. That 
is, high anxiety is associated with failure- 
avoidant interpretation. This relation- 
ship must be qualified since a testing- 
order effect was noted. When SIE was 
given first, r = —.55 (<.o1 for N = 38), 
and when SIE was given last, r = —.9 
(not significant for № = 42).* 


2. Relationship Between Avoidant Inter- 
pretation and Evaluation 

a. Absolute evaluation. In order to 
note the effect of interpretation upon ab- 
solute evaluation, the initial evaluation 
scores of 22 avoidant Ss in the lower 
quartile on the CF subscale were com- 
pared with the scores of 21 nonavoidant 
Ss in the upper quartile on the same sub- 
scale. 

As Table 6 indicates, there is a sig- 
nificant tendency for the avoidant Ss 
to give higher self-evaluation following 
the first score (a relationship which held 
throughout all six trials). That this re- 
lationship is not limited only to the ex- 
tremes is indicated by the correlation 
between evaluation score on first trial 
and criticism-failure subscale score which 
was —.21 r<.05 with N = 80). 


€ Recently we have had an opportunity to repli- 
cate this finding, but in two samples the correla- 
tion between CF score and anxiety failed to reach 


significance. 
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TABLE 6 
{ AVOIDANT INTER- 
"RIAL EVALUATION 


TIONSHIP Вет WE 
ATION AND FIRS 


| Avoidant Nonavoidant 
| Interpretation | Interpretation 


Evaluation (1-4 on CF (8-10 on CF 
| subscale) subscale) 
Low | 6 | 16 
High 16 | E 


Corrected x?=8.383; p= «.or. 


b. Over- and underevaluation. Since 
the tendency to give a high evaluation 
carries more personality-related signifi- 
cance if it occurs in relation to low 
performance, the role of avoidant inter- 
pretation was inspected as it related to 
over. and underevaluation. Following 
the procedure previously described for 
determining these two extreme groups 
of evaluators left only 29 Ss in both 
groups, which required a median rather 
than a quartile split on the interpreta- 
tional variable. 

As Table 7 indicates, the CF score is 
highly related to tendency to under- or 
overevaluate one's performance. ‘This 
tendency for the avoidant S to overevalu- 
ate his performance is consonant with 
the general characteristics of this group 
in other respects. There was no relation- 
ship whatsoever between either the PS 
subscale or the anxiety scale to over- and 
underevaluation. 


TABLE 7 
RELATIONSHIP BETWEEN AVOIDANT INTER- 
PRETATION AND OVER- AND UNDER- 
EVALUATION 


| - аа н 
Avoidant Nonavoidant 

| Interpretation | Interpretation 
(1-7 оп CF | (8-тооп СЕ 
subscale) | subscale) 


Underevalua- | | 
tors | 2 | T. 
Overevaluators | 12 


Corrected x?= 10.029; p= <.002. 


Nonavoidant Ss tended to shift their 
evaluations more (<.10) than did avoid- 
ant Ss. No relationship was noted be- 
tween avoidant interpretation and ex- 
pectancy. 


VII. DISCUSSION 
л. Avoidant Behavior 


In considering the results described in the pre 
ceding sections it appears that the behavior: 
avoidant S employs defenses which at once (4) 
distort his evaluation of his capability (setting 
excessively high goals and overevaluating ie 
performance), and (b) restrict the reception 0 
potentially useful information from the еп" 
vironmental situation (maintaining his post- 
criticism self-evaluation and making fewer shifts 
in evaluation). This defensive pattern may par 
tially account for his relative inability to Ld 
out other solutions, as well as for his ar 
rupted performance—this latter. consequence E 
sulting from the fact that he does receive 5077 
fecdback; and when such negative information 
is received, it is poorly assimilated. Such € 
ruption in transforming evaluatio [еее 
into more adaptive behavior is very similar & 
the behavior of high-anxious Ss in their pu 
tions to failure as described by Mandler anc 
Sarason (15). Restriction in the assimilation О 
information has also been described as charac 
teristic of a group of “overcontrollers” ба 
seribed by Block and Thomas (3) who герат 
such a pattern as “metastable. єп 

In light of this general relationship betwee r 
personality factors and problem-solving behave 
found here and in other investigations, it МОШ - 
scem that Subsequent research might be appro" 
priately directed toward specifying these T 
fensive patterns more specifically, Once iue 
patterns have thus been objectively delineate е 
work may then proceed on the crucial problem 
of determining the conditions which give pe 
to the development of such avoidant patterns 


В. Avoidant Interpretation 


tation 
inter 
tion 15 


The results obtained using the interpre 
measure indicate that an individual's 
pretation or perception of a negative situa re- 
an important determinant of subsequent e 
sponses. Previous attempts to measure the 2s 
ture of an individual's interpretation of a Us 
tion have been primarily carried out using pee 
rect approaches, e.g., ТАТ. That is, the analys 
of a TAT protocol, whether for clinical ОГ зі? 
perimental purposes, frequently consists of 
ferences regarding how a person perceives 
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pet Rr Whether one is dealing with 
tional ex ae рше. or subjects in a condi- 
sponses p ie s is equally true that the re- 
less the avert ыы аге not comparable un- 
way. An аео been interpreted in a similar 
which aimed А igation dest ribed by Atkinson (1) 
ties: eliciting ae determining the picture proper- 
in point E certain. interpretations js а case 
it is ee ds these considerations we believe that 
iffement. 6 Жы distinguish between the meas- 
interpret. or (а) an individual's tendencies to 
picture, and Pre a situation, eg. а ГАТ 
event once ic |, his available reactions to this 
way, That is ee been interpreted in a given 
vidual's ae : опе wanted to measure an indi- 
should nes ey organizational tendencies, he 
stimulus: ка, эу select a highly ambiguous 
ess the eem it the investigator desired to 
it might be Маш s reactions to certain events, 
highly АШ уе Appropriate to select more 
Procedure — stimulus materials. This latter 
Procedure =. ШЕ exemplified by a hypothetical 
Гау series hee "aah Card GBM in the Mur- 
Structuring ке mother son" card) with the 
his iob and селер, This man has just lost 
It should be is talking to his mother about 10. 
any necessar made clear that we do not assume 
interpretation Sne:toxne relationship between 
maintain qu and behavior, but rather would 
Situation Ш. the interpretation of the 
controlled) that d measured (and, if possible, 
More accura уз the behavior may be studied 
pte ately, 
dimensional к use of a forced-choice single 
is limited Fa M for assessing interpretations 
adequacy-ina 95, the reference system employed 
Not be Тац» шор in the present са se) may 
culty is са у appropriate for all Ss. This diffi- 
tive prci enced by the relative lack of predic- 
9S at the ee it was necessary to split 
see Table «ап on the interpretation measure 
CY of 55 A However, in cases where the num- 
1р5 (e permitted the use of extreme avoidant 
ашиде үрк SPB ав in Table 5) the re- 
Words, it pu was greatly increased. In other 
re in me likely that for those Ss who 
cing employ 3-7 range, the reference system 
yed is not especially relevant. How- 


eve 

» th е 

Should le methodological paradigm of the SIE 

that oth nd itself to studying the probability 

Will authority situations, 
ms, 


the 
ons 
d with 


Score 


е Brakes the positive results reporte 
“Success subscale (Section V)- 


Ww 
Person ave also briefly noted the effect of the 
hat is the agent in the situational stem. 
etation рго- 


the differences in interpr 


duced by peer-induced and adult-induced criti- 
cism suggest that the method might be useful 
for measuring indirectly the meaning or signifi- 
cance which a given figure in the situation may 
have for one individual or for a group. ` 

The implications of Sections V and VI to re- 
search in the area of individual variation in 
reaction to criticism, failure, praise, and success 
may be briefly considered. Experimenters in the 
area of social motivation have for some time 
been concerned with the fact, noted by Grace, 
that “that which might act as blame for one 
individual might act as praise for another” (о. 
p. 77) Utilizing constructs such as introversion- 
extroversion (26) and ascendancy-submission (25) 
empts have focused on accounting 
for performance variation, implicitly assuming 
that some change in self-evaluation has inter- 
vened. In using the present approach anchored 
in this intervening process, we have indirect evi- 
dence (Sections II, V) to support the contention 
that Ss who refuse to lower their evaluation fol- 
lowing failure or criticism perform more poorly. 

That interpretation scores on the criticism- 
failure subscale were associated with initial ab- 
solute evaluation (Section VI) is not surprising 
when it is noted that this evaluation followed an 
initial, fairly low score which may have been 
interpreted as failing. These initial differences 
in absolute evaluation masked any hypothesized 
changes in relative. evaluation—which leads to 
the speculation that the interpretation variable 
appears to have its maximal effect in reaction 
to the first cue provided, in this case, a low 
score. A similar primacy of effect has been noted 
in connection with differences in response at- 
tributable to an anxiety variable (16), in that 
differences between anxiety groups were "di- 
luted” after the first tri 1. 

In considering the findings from Sections V 
and VI it is important to recall that in the 
former, 5 received no scores ОГ other evalua- 
tional information until E interjected a com- 
ment, and prior to this, there were no differ- 
ences between subgroups in absolute evaluation. 
In Section VI when scores were given, differences 
in evaluation Were present from the outset. In 
attempting to synthesize findings from these two 
studies, it would appear that the nonavoidant 
Ss arc sensitized to the initial evaluational cues 
in a new situation whether such cues come from 
their scores or from E. This heightened sensi- 
tivity characteristic of nonavoidant $s may be 
viewed in contrast to the previously described 
defensively-restrictive characteristic of the avoid- 
ant Ss, In overevaluating their performance 
(Section VI) the avoidant Ss bear a certain re- 
semblance to the overly confident Ss described 
by Block and Petersen (9) as rigid and dog- 
tic. Conversely а certain similarity in person- 


previous att 


ma 
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ality pattern is noted between our nonavoidant 
Ss and the overly cautious Ss described by these 
authors as tending toward introspection and 
self-abasement. The present referents for avoid- 
ant and nonavoidant interpretation might also 
be considered as reflecting different means of 
dealing with blame-assignment (17). 


C. Relationship of Avoidant 
Interpretation to Intelligence 


The relationship between low intelligence and 
avoidant interpretation must be qualified. But, 
even if such a low-order relationship is verified, 
this does not necessarily detract from the value of 
the interpretation measure, That is, it may well 
be true that one of the possible antecedents for 
the adoption of avoidant interpretation is a de- 
ficiency in problem-solving ability. That intelli- 
gence is not the only factor involved in avoidant 
interpretation is to be noted in the fact that it is 
not related to behavioral avoidance (Section IV) 


D. Concluding Comments 


From a conceptual point of view our findings 
represent what Cronbach and Mech] have called 
a “nomological net” (7) which is anchored pri- 
marily in the behavior of failure-avoidance and 
secondarily in the behavioral measure of self. 
evaluation. The present results are implicit evi- 
dence for the value of considering the psycho- 
logical situation in attempting to predict the 
reactions occurring in that situation, 

Certain shortcomings and limitations of the 
present approach should be mentioned. In our 
present exploratory attempt to measure inter- 
pretations we have used a relatively imprecise 
instrument, with the disadvantage that the 
avoidant alternatives are not necessarily homo- 
geneous regarding defenses, We have not dem- 
onstrated any independent criteria 
ing the utility of failure-avoidant behavior al- 


interviews indi- 


The reader has probably noted that in our 


attempt to demonstrate interpretation-produced 


SCHRODER AND DAVID F. 


HUNT 


pectancy (18) and their combined pues hk 
problems deserving further concerted investiga 
Б" illustrate. the conceptual complexity К 
this problem, Atkinson's recent description me 
attempt to measure fear of failure by er 
imaginative productions may be noted (1). to 
though this fear-of-failure measure diae “ 
be operationally similar to the present perme (st 
interpretation measure, Atkinson conceptual à 
this construct as a "negative" motive with PER 
erties opposite to those of achievement er 
We cite this example not as an “opposi g 
theory,” since we do not feel justified in ae 
ing a complete theoretical conception of failu x 
avoidance at this time, but rather to point zu 
the necessity for further investigation. We E^ 
strongly, however, that such investigation. m 
quires relevant behavioral anchor points a 
order to develop theoretical conceptions for ү 
understanding of avoidant processes, In iat 
present investigations the measures of pug ens 
behavior and self-evaluation appear to ha 
served this purpose. 


УШ. SUMMARY 


A series of hypotheses derived from om 
problem-solving behavior and eB 
tions of individuals employing avoidan 
mechanisms were tested and generally 
supported. These findings, which appes 
to be consonant with general koe ed 
tions regarding the operation of " 
sive processes, may be summarized. 
terms of two constellations of relation 
ships, — 

The Ss who utilized failureavoidar 
behavior in a problem-solving iuam 
were found to: (a) set higher goals, ie 
use fewer alternative solutions in (0 
tempting to solve the problem, and С 
perform less effectively after failur d 
These relationships were noted in pn 
identical studies using male college Ке 
Following leads which emerged from ee 
postexperimental interviews, a meast " 
of an individual's tendency to avoid oig 
preting negative situations in terms 
personal inadequacy was constructed. Д 

In employing this interpretatio 
measure with high school boys, Ss W 
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. interpretations of failure 
failure ge wine found to (a) avoid 
(b) so ci cima mi situation, 
и течи " E я weir self evaluation after 
loving "e 4 inp a evaluations fol- 
мыл, . ow failing score, and (d) over- 
"rt" their performance, 

Bite o dude generalizations must be 

to college and high school male 


on separate studies using similar meas- 
ures, and in at least one case—behavioral 
avoidance and high goalsetting—we 
were able to replicate this relationship 
in both populations. These relationships 
appear to integrate a number of experi- 
mental and clinical observations between 
personality characteristics and problem- 
solving behavior under negative condi- 


opulat; е 
Populations studied, the results are based tions. 
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d Wes literature regarding auditory fa- 
А Ligue is abundant. Without attempt 
ing an exhaustive review of the litera- 
ture, the studies of Caussé and Chavasse 
(3). Davis et al. (5), Epstein (7), Harris 
Es 11, 12). Hirsh and Ward (14) 
та (22), Rawnsley and Har- 
eee and Rosenblith (25), Rosen- 
Rosan BO and Hirsh (26), and 
ав ое апа Rosenblith (27) show 
o much effort has been exerted to re- 

€ such factors as the frequency, 1nten- 


sity : А 
н. and duration of exposure of test 
ш. auditory 


i poststimulatory 
thresholds, 


jop oe has appeared indicating that audi- 
ie., шау effects сап be bidirectional, 
Sensitivity an increase or a decrease in auditory 
tion, Sven can follow intense auditory stimula- 
Come p. investigators have reported what has 
12, 18 е called “sensitization” (1, 2 4 5 8, 
here dane 15, 16, 21) following stimulation, It is 
t ШОШ аз a lowering of the normal auditory 
auditory (or conversely, an increase in DAC 
Teport -A acuity). Some investigators (1, 18, 16) 
9f the “з Sensitization varies with the frequency 
Posure n tone or with the frequency of the ex- 
ы SAM Bronstein (1) found that sensitiza- 
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tion was less for low frequency tones and increas- 
ingly greater for higher frequencies, Hughes (15) 
noted maximum sensitization at 300-500 cps. Не 
also reported that one of his three extensively 
trained subjects showed sensitization to a pure 
tone after a white-noise stimulus. 

One theoretical formulation has been offered 
to explain sensitization. Hughes suggested post- 
tetanic-potentiation® as a possible answer. He be- 
lieves that the continued stimulation of the audi- 
tory system by the exposure sound tetanizes the 
relevant nerve fibers, provided the sound is of 
sufficient intensity. Eccles (6) found that con- 
tinued tetanization of nerve fibers increased the 

oststimulatory nerve potential. Hughes con- 
cluded that sensitization was due to an increase 
in poststimulatory potentials from the stimulated 


auditory nerve fibers. 


If post-tetanic-potentiation is a valid 
explanation for sensitization, then a 
broad-band noise source used аз ап ех- 
posure sound at an appropriate intensity 
should result in sensitization for tones 
within the frequency limits of the noise- 
band stimulus. On the other hand, if 
the noise were Гей through the proper 
acoustic filters and a narrow gap re- 
moved from it, there should be less or 
perhaps no sensitization for frequencies 
within the filtered gap because less tet- 


anization should result in a decreased 


sensitization. 


Ss 
a Eccles (6) defines post-tetanic-potentiation 
(PTP) as the “... relatively prolonged increase 
in response that occurs after a junctional region 
has been subjected to repetitive orthodromic 
activation. . + ." In other words, PTP is an ob- 
t increase in neural activity at a 


served, persisten se in à 
synapse following tetanic stimulation 
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If a filtered noise is used as an ex- 
posure sound and pre- and postexposure 
thresholds are determined for tones with- 
in the filtered gap, reasoning suggests 
that the receptors associated with the 
frequency area within the filtered gap 
would receive less stimulation. during 
exposure and, therefore, be characterized 
by a different poststimulatory response 
than for those of frequencies outside the 
filtered gap. An unfiltered noise stimulus 
could serve as a control for testing the 
above hypothesis. 


The purpose of the present study was 
to determine the effect of filtered and 
unfiltered noise stimuli at two sensation 
levels upon the poststimulatory thres- 
holds for pure tones located in and out- 
side the filtered gap. 


I. APPARATUS 


The noise stimuli were recorded on high- 
fidelity magnetic tape using an Ampex 600 series 
recorder. The recording was made at a tape 
speed of 15 inches per second. The white-noisc 
source was a Gerbrands Noise Generator, PAL., 


1 KROHN-HITE MODEL 440-A 
OSCILLATOR 


2. AMPEX MODEL 400 TAPE 
RECORDER 


3. R.W. CRAMER INTERVAL TIMER 
4. PENER MODEL 50C PRE-AMPLI- 


5. FISHER MODEL 50A AMPLIFIER 
6. CLICKLESS SWITCH 
7 


. HEWLETT-PACKARD MODEL 3 
ATTENUATOR = 


8. WESTON MODEL 769 VTVM 
9. DUMONT TYPE 218 сяо 


10. UTC 534 MATCHING TRANS- 
FORMER ® TRANS 


11. PERMOFLUX РОВ -1 
WITH DOUGHNUT CUSHIONS 
12. LANGE OF LEXINGTON, WAXED 


PAPER RECORDER AND HAND 
HELD SWITCH 


О EARPHONES 


HAND HELD 
SWITCH 


Fic. 1. Block di 


EARPHONES 
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type 422. A gap extending from 100 to bon Ex 
was filtered from the noise by passing the signa 
through two series-connected Kiohn Hite 850-5 
ultra-low-frequency rejection filters which Es 
placed ahead of the recorder. ‘I һе filters: w am 
calibrated by a series of pure-tone input signa 3 
from a Hewlett Packard model 200€. audi 
oscillator. 

A block diagram of the apparatus 
shown in Fig. i. 


used 15 


П. SUBJECTS AND PROCEDURE 


Seven subjects (Ss) were used. v e 
the Ss but one had had prior experience 
in attending to auditory stimuli in 7 
perimental studies. All Ss had normit 
monaural acuity (best ear). Normal Б 
here defined as acuity of not more Ben 
10 db below normal at the octave vi 
quencies 125-8000 cps. Ап otological pa 
amination was given all prospective ad 
and none was accepted who showed pd 
evidence of throat pat! 


ear, nose, or 


alance test 
ology. An alternate loudness balance * 
^ 4 а OI 
Was used to screen for recruitment. E e 
б » å М vo те 
of the Ss used in this study showet 


cruitment. 


CLICKLESS 
SWITCH 


ATTENUATOR 


MATCHING 
TRANSFORMER 


RECORDER 


agram of apparatus. 
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АП testi 
with Pia уы аш 3 in a sound-treated room 
Ss wore йрй ambient noise level of 45 db. 
Provided about = with doughnut cushions which 
mental, sound Pe db attenuation of environ- 
termined ite rus, the Ss’ thresholds were de- 
ambient бе ж га 25 db background (45 db 
tion), minus 20 db carphone attenua- 


The seri; 
Mine ай АН of limits was used to deter- 
4 crossings, з a р ds, Each threshold was fixed by 
The Ss qu Sending and 2 descending series. 
aural Sue d quM monaurally and mon- 
Familiarizati БЕ taken. 
before the "Доп training was given to cach $ 
Consisted aaa was begun. This training 
determine t 3 complete audiograms, 5 trials to 
Noise Goel Ls for the study, and 2 white- 
НЕ ШШ determinations. 
the pe Дн 5 came into the testing room 
Was seated T NEC placed upon his head and he 
to be expose i s threshold for the test-tone he was 
then determi to for the experimental session was 
minutes to печ and he was exposed for 4 
At the end з filtered. or the unfiltered noise- 
noise was е the 4-minute exposure period the 
Switched on minated, the test-tone circuit was 
mining апе the experimenter began deter- 
lest-tone S's postexposure thresholds for the 
ў on of the vari- 


The order of presentati 


ables 

S OF intensi 4 
lered or intensity, test-tone frequency, and fil- 
ized, unfiltered noise exposure Was random- 


д ПІ, EXPERIMENTAL VARIABLES 
ва вада levels were used, one 
The 4 im the other of 20 db SL. 
Posure $ were exposed to 2 different e~- 
tered n a filtered and an unfil- 
filtered The filtered noise had а gap 
requen тош 100 to Goo eps with a center 
db, The. maximum attenuation of 25 
оаа Ба unfiltered noise stimulus B 
BEE noise with a fairly 
Noise ә Duration of exposure to me 
in e stimuli was | minute with 4 hours 


lites s. 
si Srvening 1 RR ital ses- 
Stong etween experimet al Se 


of 


as 


TI 
Ne тек 
est- Р 
TN сопе frequencies were as fol- 


s 
1 on 
frequen € tone of 255 cps as the center 
C " 
Center р? (which corresponded to the 


Tequency of the filtered gap), а 


ton 
e 
vm m cps which was above the 
Vhich quency, and one of 162 CPS 


el 
l below the center frequency: 


RELATIVE SOUND PRESSURE LEVEL 


FREQUENCY 


. Calibration of the filtered noise. 


The 162- and 350-cps tones in the filtered 


gap were attenuated approximately half 
as much as the 255-cps tone. Then signals 
of боо-срѕ and 100 CPs were selected. 
signals were chosen because they 
upper edges of the 


ne frequencies were 


These 
fell at the lower and 
filtered gap. Testtor 
selected by examining the calibration 
curve of the filtered noise (see Fig. 2)- 
Five postexposure thresholds were 
taken within time intervals of 45 sec 
o minutes, 4 minutes 


onds, 90 seconds, 2 
and 8 minutes after termination of the 


exposure sound. 
IV. RESULTS 

The major finding of this study is that 
Ss stimulated with the unfiltered noise 
consistently had higher postex posure 
thresholds for pure tones within the fil- 
tered gap than did those who were stimu- 
lated with the filtered. noise. This sug- 
t the unfiltered noise produced 
fatigue while the filtered noise 
n sensitization. The above pos- 
d by the paired replicates 


test (23): Results of the statistical analysis 


of the data may be seen in Table 1. 
and 5 present the graphical 


gests tha 
auditory 
resulted i 
sibility Was teste 


was prese 
to all test-tones was 1 
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TABLE 1 
STATISTICAL ANALYSIS OF DATA 


А. 2X2X5X7 Overall Analysis of Variance тет 
Ап analysis of the effects of 2 types of noise (filtered and unfiltered), 2 intensity levels, and 5 ditte 
test-tones upon the postexposure thresholds of 7 Ss. 


Sum of Mean 


Source of Variance df Squares Square es ا‎ 
E EXE TS 2.00** 
Filtered vs. Unfiltered Noise...............- I 1408.11 E 
Тепа буе sis алг are ил 1 584.25 ddr 
Frequency....... 4 84.40 : аж" 
Е. vs. Unf. XIntensity... 1 460.84 m + 
F. vs. Unf. XFrequency . . 4 80.70 б 
Intensity ХЕЁгедиепсу................... 4 30.30 p aber 
BA GC IDEST ee atre син. sex a: ты na а г 4 50.20 9-20 "— 
B. 2X5X7 Analysis of Variance of the тоо db Data m-— 
Sum of Mean А 
Source of Variance df Squares Square F = 
си u— B À —À ж+ж 
1 1740.00 1740.00 70:73 44 
4 400.60 128, 18 12.92 4 
4 548.70 137.10 pe^ 
6 147.70 24.60 СРЕ 
С. 25 Х7 Analysis of Variance of the 20 db Data ШЕЕ. 
А Sum of Mean F 
Source of Variance df Squares Square и... 
" = ہے سے‎ 
Filtered vs. Unfiltered Noise 1 128.04 128.04 18.14 


* Significant beyond the .or level of confidence. 
* Significant beyond the .oo: level of confidence. 


Stimulation by unfiltered noise at зо db 
SL resulted in significant fatigue only for 
the test-tones of 100 and боо cps (Fig. 3). 
Stimulation by the filtered noise at 100 
db SL resulted in statistically significant 
sensitization for the 162- and 255-Cps sensitization are greater following | ue 
test-tones, while fatigue was found for db SL stimulation, than go db SL sti 
the 100- and 600-cps tones (Fig. 5). At the lation (Fig. 5). 

зо db SL, statistically significant sensiti- The temporal course of postex] 


А was found for only one test- thresholds was shown to vary as a fun uli 
that of 350 eps. Signal detection was of whether there was initial poststin? 1 
found to be significantly better after fil- 


Ч РОМА: А tory fatigue or sensitization. Where р a 
tered noise stimulation than fol stimulatory fatigue was found, the Ja 
unfiltered noise exposure, tigue tended to decrease with time ( Ёз 

Generally, the fatigue effect was more Figs. 3 and ). An analysis of the r€ 
prominent and more consistent than the of the мач. реу showed that ur 
sensitization effect. The fatigue effects sensitization on observed initially: e 
for the unfiltered noise were essentially next threshold was more sensitized i” ibe 
the same for all the test-tones while the records of four aie of six Ss. In on€ ott 


sensitization effect following filtered 
noise stimulation tended to он nr 
filtering effect, This may be seen by p» 
paring Figs. 5 and 6. 


т 
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Results indicate that fatigue a? 
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Fic. 3. The pooled data for all Ss, showin 
thresholds for the 20 ¢ 


mors anion cases which did not re- 
Silas. as above, all jus posue thresh- 
The bs at essentially the same level. 
Ey Ea sensitization example showed 
sitivity ter the initial sensitization, sen- 
меои „бени, increased, then de- 
роу again. The temporal course of the 
Xposure thresholds is shown by Figs. 
запа 4. 
Ten the course of the ех 
їйїр Ы 1e seven Ss consistently reported 
tion p following 100 db SL stimula- 
experi t the end of the experiment the 
Pitch a asked the S to match the 
tor жү is pure tone from a signal genera- 
tinnit the frequency of his tinnitus. The 
us following the unfiltered noise 


periment 


— © 


osure 


g the temporal course of the postexp 
ib SL stimulation. 


was generally of a higher frequency than 
that following the filtered noise. In every 
the subjective tinnitus was of a 
the frequency of the 
sed in the study. 


case, 
higher pitch than 
highest test-tone U 


V. DISCUSSION 


It was found that thresholds for tones 
within the filtered gap were lower follow- 
ing filtered-noise stimulation and higher 
following unfiltered-noise stimulation. 
Hughes (15) explained sensitization as 
due to post-tetanic-potentiation. The re- 
sults of the present study are opposite 

would be predicted on 


those that 
the basis of post-tetanic-potentiation. 


The pos t-tetanic-potentiation hypothesis 
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Fic, 4. The pooled data for all 5з, showing the temporal course of the 
postexposure thresholds for the 100 db SL stimulation. 


would predict that more sensitization 
should result after the unfiltered noise. 

The results of this study, though not 
explained in the literature, might be ex- 
plained in a formulation borrowing from 
Tanner and Swets (29) their ideas re- 
garding the hypothetical distribution of 
noise and signal-plus-noise for intensity, 
and their philosophy that the problem 
of detection is one of the detection of 
signals in noise. 

It may be assumed that any sensory 
channel is continuously activated. The 
origin of this activity, hereafter referred 
to as ambient noise, may be ambient 
stimulation, neural noise, or both. 


The ambient noise is assumed to 0 
proximate white noise. By definition, 2 
a given frequency the intensities whit 
would result if the noise were continuo 
would be distributed normally about i 
mean intensity which is the intensity në 
the noise. Assume further that pe 
listener can discriminate perfectly e 
qualified later) among signals! of dif “ 
ent frequencies, and in an experime 


oer’ 

‘The term signal is used to refer to an e ord 
mentally produced stimulus while the w! піс! 
Stimulus is used to mean any stimulation ier 
activates the auditory system of the S. rhe py 


А € А ation 
stimulus does, of course, include stimulat 

1 ‹ А 
ambient noise. 
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о sen- 
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sation es Unfiltered noise stimuli at OW 

boola] Pu (This figure is derived from 
data of Figs. 3 and 4.) 


Is 
ld ple: qa of only onc fre- 
The tan, of the test-tone. | 
ыбы О noise and signal: 
beuen distribution for intensity and 
When y may be seen in Fig. 7: u 
“ signal] he S is instructed to listen lor 
ог not a пе attempts to decide whether 
а signal is present. At all ti 


Stim 

ulus 5 

Us of the expected frequency 
t of the 


es at a 
ributed, 
a given 


mes à 
Prese is 
Sent since it | 

апе 2 леп 
Mbient it is a componer 
Biven freq 
When the 
i 

Ntensity 1 


noise, Since intensiti 
uency are normally dist 
S gets a stimulus of 
like he must decide whether it is 


1 | 
no) that the stimulus came from the 


Oise 
Se that j 
lat is always in the system. Аз the 


i 
Density À ip 
of the stimulus increases it 
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Fie. 6. Calibration curve for the filtered noise 


om a linear scale to facilitate comparison with 
the fatigue and sensitization curves). 


moves the signal-plus-noise distribution 


farther out along the intensity axis. The 
ise distribution is unchanged by the 
al; therefore, it becomes 
that the S will say 
imuli are coming 


no 
addition of a sign 
increasingly less likely 
that more intense st 


Irom the ambient noise. 
The following alternative exists. If 


N S+N 


PROBABILITY 


(А) 
INTENSITY 


PROBABILITY 


(B) 


ROM THE TEST TONE) 


gency (IN GPS F 
GRITERION 
CRITERION 


ns of noise (N) and 
sity and fre- 


FREQU 


(A) INTENSITY 
(B) FREQUENCY 
T: jstributio 
Fic. 7- The distrt » 
signal. pluscnoise (5+ № for inten 


quency: 
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the experimenter presents a ees 
given intensity to the 5 the total d = 
bution of intensities of a signal of that 
frequency changes, since the intensity 
distribution then has the ambient noise 
element as well as the signal component. 
If the signal is above the intensity level 
of the noise, the mean of the signal-plus 
noise distribution will be higher than the 
mean of the noise distribution. In the 
absence of a signal, the S samples only 
the noise distribution. But when a signal 
is introduced, the $ samples а signal-plus- 
noise distribution that differs from the 
noise distribution. Therefore, higher in- 
tensity stimuli are more likely from the 
signal-plus-noise distribution. 

Although the S is instructed. to re- 
spond in some way when he he 
cific signal, his actual task, if this explan- 
ation is correct, is to decide which of the 
two distributions he is sampling. The 
better guess which the S can make results 
in his responding "no signal" when he 
gets a stimulus that is closer to the mean 
of the noise distribution than to the 
mean of the signal-plus-noise distribu- 
tion. Stated in different terms, the § 
establishes for himself an intensity сгі- 
terion and any stimulus above this cri- 
terion will elicit “signal,” while stimuli 
below this criterion will elicit “по sig- 
nal,” 


ars a spe- 


If, as has been assumed, the two distri- 


utions are normal and have the same 


b 


actly 
ans of the two 
|t may be that he 
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Assuming that the 5 cannot disertis 
nate perfectly among frequencies (or ет 
the signal generator produces not олу 
the desired frequency but also neigt 
ing frequencies) and assuming pr 
ously as has been previously shown) de 
intensity is constant, it is possible, by " 
argument similar to that above, to Hips 
at the following noise and signal-plus 
noise distributions for frequency. re: 
cause of the physical nature of bu 
noise, the distribution of шана Я 
rectangular, i.e., all audio-frequencies g^ 
present and at any moment cach d 
equally likely to appear. But the sign: " 
plus-noise distribution for ари 
not flat. The signal generator is assum ч 
to put out more signals at the a 
frequency but it also puts out othet go 
nals that distribute Шишек nari d 
around the mean frequency which 1s t 2i 
of the test-tone, It should also be on to 
that the $$ brought in and expose " 
his test-tone while his pre-exposure yas " 
hold is determined; therefore, he know 
what signal to expect. But since wd 
cannot discriminate perfectly ноа 
frequencies Һе is more likely to ee 
"signal" to a tone of the same freque! 


З ; Jess 
as the test-tone and increasingly as 
= "m ” es 

likely to respond "signal" to ton test 


they differ in frequency from the Jus- 
tone. If this were true, the ШЕРТ 
noise frequency distribution ош 
symmetrical about the modal ec 
the probability of a response to up 
lus would be decreased as the stim re 
differs in frequency from the center ne) 
quency (which is that of the pem 
regardless of whether the frequency [ro 
ference is higher or lower with ер, to 
the testtone, The S sets some limi the 
the amount which the frequency 9 cest 
stimulus may differ from that of ee pis 
tone and still be called "signale јол 
may be called а frequency a а 
hus, any stimulus within the freq" 
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criterion will be called “signal” and any 
outside this limit will be “no signal.” 

е If the intensity and frequency assump- 
tions made above are combined, omitting 
the erroneous assumptions of intensity 
and frequency constancy that were made 
for convenience, а three-dimensional 
graph of the probability of occurrence 
of a stimulus of given intensity and fre- 
quency can be drawn. Probability of a 
response can be determined once the 
probability of a stimulus' occurring is 
known. Figures 8 and 9 represent three- 
dimensional graphs of the probability 
of occurrence of a stimulus of given fre- 
quency and intensity. 

Examination of Figures 7, 8, and 9 
shows that part of the noise distribution 
15 included within the region delimited 
by the 55 intensity and frequency cri- 
teria. This occurs since the two distribu- 
tions (noise and signal-plus-noise) over- 
lap and the S's criterion would normally 
include some of the noise distribution. 
Also, not all of the signal-plus-noise dis- 
tribution is included within the S's 
criterion, This means that some noise 
stimuli will be called “signal” (a “false 
alarm”) and some signals will be missed. 

An additional assumption necessary in 
ШЕ above formulation is that auditory 
Stimulation fatigues the S at the stimu- 
lating frequency and to neighboring fre- 
quencies in a gradient fashion with the 
Maximum fatigue occurring at the test- 
"one frequency. Figures 8 and 9 show 
that stimulation at frequencies other 
than that of the test-tone would affect 
the Sensitivity of the 5 in such a manner 
Шш the part of the signal-plus-noise dis- 
tribution away from the test-tone would 
be maximally affected. The altered sen- 
Sltivity of the $ would also result in the 
оне distribution’s being changed. The 
har belief underlying the present for- 
tl ulation is that the relative volume of 

1€ noise and the signal-plus-noise dis- 


tribution included within the area de- 
limited by the intensity and frequency 
criteria determines auditory response. To 
elaborate, if an S is stimulated by a given 
signal, his sensitivity decreases for a band 
of stimuli. The stimulating frequency is 
at the center of the depressed band, the 
frequencies on either side of the center 
frequency are also depressed but to pro- 
gressively lesser degree. If, after having 
been stimulated, the S has his threshold 
determined for a given tone for which 
his normal threshold is known, the 
change in sensitivity following stimula- 
tion can be observed. The above formu- 
lation holds that the stimulation would 
change the S's sensitivity and therefore 
change the volume of the signal-plus- 
noise distribution relative to the noise 
distribution included within the criteria- 
passing region. If the postexposure signal 
were at the center frequency of the stimu- 
lated band, sensitivity would be maxi- 
mally decreased. This would occur be- 
cause the frequency signal-plus-noise 
distribution peaks at the signal frequency 
and drops off sharply thereafter. The 
noise distribution for frequency, on the 
other hand, is flat throughout its range. 
Therefore, a depression of the signal- 
plus-noise frequency distribution at the 
signal frequency removes relatively more 
signal-plus-noise than noise distribution. 
This change in the relative volume of 
noise and signal-plus-noise should be 
manifested as fatigue. 

It is believed that essentially the op- 
occur when the $ is stimu- 
lated by signals other than at the test- 
tone frequency. Stimulation by fre- 
quencies other than the test-tone fre- 
quency would affect the signal-plus-noise 
distribution in its less peaked area and 
would affect the noise distribution as 
before, in a flat area. Therefore, more 
noise and less signal-plus-noise volume 


posite would 


p 
PROBABILITY a | 


re, 


INTENSITY 


aph of the noise (№) and signal-plus-noise (S + N) 
the overlap between the two distributions. 


Fic. 8. A three-dimensional gr 
distributions, showing 
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оп 
‘ istributio™ 
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showing the area delimited by the intensity and frequency criteria. 


PURE-TONE 


would be removed and sensitization 
would result. 

However, the above formulation lacks 
sufficient generality to account for some 
of the previously cited sensitization re- 
sults, Specifically, the above explanation 
Predicts fatigue following stimulation at 
the test-tone frequency, while the pre- 
Viously cited studies report епха бта 
= een that the present formula- 

nust be modified in order to ac- 
count for these results. 
ыс modification proposed is of the 

ing nature; In this study and in 
that of Hughes (15) some Ss reported a 
n PD tinnitus. This postexposure 
Mate ч could be assumed to approxi- 
а m. frequency of the test tone and 
Kita the shape of the noise dis- 
longer ka for frequency so that it 15 ЛО 
Irequer T but peaks at or about the 
curred pe of the tinnitus. If this oc 
followe ; hether fatigue or sensitization 
mined з ета would Ье deter- 
пове к the relative peakedness of ui 
Bins ‘signal-plus-noise frequency sl 
signal. ei А more peaked noise ап 
sult T EOE distribution would el 
vious д с ао {following the pe 
tributi rgument) while if the noise dis- 
ens on were less peaked, fatigue would 

ue, 

аы of jns — — 
Cussed, Ur a frequency have been dis- 
tion foll nlike Hughes's study, sensitiza 
Was fo owing a low level of stimulation 

TI und in the studies reported here. 
Жене e study reported a деши 
that hpi function suc 1 
below a and exposure frequencies 
ете above the range 300-500 CPS 

d different results from fre- 

Of Figs s within that range. Examination 
Study tn, and 6 show that in the yos 
ere Ne frequency-sensitization е Ens 
€ similar to the filtering curve. 
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From this it is inferred that differential 
frequency effects in the range spanned 
by this study were small or nonexistent. 
1f there had been noticeable differential 
[requency effects they should have made 
the frequency-sensitization curve deviate 
markedly from the filtering curve. 
Figure 5 also reveals that fatigue effects 
for the different frequency test-tones were 
about equal. These results are consistent 
with those of Caussé and Chavasse (4), 
who report that in the frequency range 
of 100-600 cps spanned by test-tones in 
this study, little or no fatigue effects can 
observed. Several further studies are 
the results of this study. 


be 
suggested by 


VI. SUMMARY 


Pure-tone prestimulation thresholds 
were found for Ss who were then exposed 
for four minutes to the filtered or un- 
filtered noise stimuli at either a 20 or 100 
db sensation level. A series of post- 
exposure thresholds were then deter- 
mined for the same tone for which the 
pre-exposure threshold had been found. 
‘The рге- and postexposure thresholds 
were then compared. 

The Ss stimulated with unfiltered 
noise consistently had higher post- 
exposure thresholds for pure tones with- 
in the filtered gap than did Ss exposed 
to the filtered noise. Following a 100 db 
SL stimulation by unfiltered noise, fa- 
tigue resulted for all test-tones. Follow- 
ing the go db SL only the 100- and 
Goo-cps tones were fatigued. On the 
other hand, а 100 db SL stimulation by 
filtered noise produced sensitization for 
the 162- and 255-cps_test-tones and fa- 
tigue for only the 100- and Goo- cps tones. 
or the filtered noise at the 20 db SL, 
as found only at 350 cps. 
poststimulatory sensi- 
Itered noise were 
ith the relative 


For 
sensitization W 

Variations in 
tivity following the fi 
«n to be correlated w 
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position of the testtones with regard 


to 
if 


the filtered gap. Results indicate that 
fatigue follows 20 db SL stimulation, 


considerably more fatigue may be ex- 


pe 


cted after 100 db SL stimulation. The 


same trend appears for sensitization. 


better 


th 


The Ss were found to detect signals 
after filtered-noise stimulation 
an after unfiltered-noise stimulation. 

Within the limitations of this study 


the temporal course of postexposure 
thresholds was not found to be of a def- 
inite cyclical nature. 
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йж of color vision are based 
КА E on results of experiments 
fis ч ат spectral luminosity, 
ШЕ Тыт ас ар!айоп апа contrast, and the 
н igth discriminations of normal 
ч color-blind subjects, These sources 
ur езй а large amount of informa- 
La к; have failed to provide conclu- 
se TER on such basic matters as the 
and tke о undamental response curves 
S spectral characteristics of each. 
S of the failure of traditional 
Shan. ind to the problems of color 
bus а мна seem desirable to follow 
NALH mising new leads that become 
ing of е. Among NE are the record- 
ниш а electrical responses (electro- 
js еее іп the human eye when It 
engths, ar А by lights of various wave 
the ета . microelectrode studies of 
Certain a single retinal units in 
nal preparations. Unfortu- 


nate] 
tine] E spectral sensitivity data ob- 
Microe] Y Granit (7, 8) and others by 


lave E ci techniques of recording 
uman eye the first place involved the 
lave bean p. nd in the second place they 

ased on somewhat indirect 


ale, 

"ulatio ; 

Tom ibe ns. This follows inevitably 
fact that recording was from 


“This s 

ological Тарп was conducted in the Psy- 
Om Supported ratory of Brown University an 
ane aval by a contract between the U 
naow at th Research and Brown University. 
ayetsity. € Department of Psychology, Yale 
toe OW, 

Bi New ће U.S. Medical Research Labora- 

Ondon, Connecticut. 


third-order neurons, each of which was 
activated by a combination of numerous 
receptor cells. The human electroretino- 
gram is even more a reflection of mass 
electrical activity; and while it has 
yielded some evidence for individual 
color response processes, it is far from a 
direct. measure of the spectral char- 
acteristics of each. 


Tue MOTOKAWA EXPERIMENTS 


A new and different method of relat- 


ing retinal phenomena to problems of 


color vision has recently been described 


by Motokawa (6, 17 18, 19). This 
method is one in which the human eye 
is stimulated. briefly by an electric cur- 
rent, using silver electrodes in contact 
with the skin of the brow and cheek. 
Currents of a few tenths of a milliampere 
are sufficient to stimulate the retina. 
They give rise to “phosphenes,” which 
present an appearance to the subject (S) 
of vaguely defined clouds of light in the 
1 field of view. An electrical 
is determined by reducing the 
ive presentations of the 
the S reports that the 
longer aroused. The 
d so determined is 
y the state of 
nd by other 
Of particular interest to 
e observation that a flash 
ng before the electrical 
to increase the 


periphera 
threshold 
current in success 
test pulse until 
phosphene is no 


electrical threshol 
found to be influenced b 


variables. 
Motokawa is th 


of light, occurri 
delivered, acts 


ulse is 1 s 
sensitivity of the eye to electrical sti т 
2 “сч пеѕеа" с 
pure?" ganl ^uo qari EE 
D fant 


pated m 
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tion. This suggests that the electrical 
threshold can be used as a measure of 
the excitability of the eye, so that the 
effects of light on the eye may be evalu- 
ated by measuring the alterations that 
the light produces in the electrical thresh- 
old. Specifically, Motokawa has under- 
taken a series of studies in which a flash 
of light has been followed at a given 
interval by a test pulse of direct current. 
A threshold is determined for the stimu- 
lating current, on the basis of its ability 
to arouse phosphenes in the stimulated 
eye. A comparison is made between the 
phosphene threshold following a light 
and the phosphene threshold in the dark. 
In this way a measure is obtained for the 
enhancement of the phosphenes by the 
preceding light. Since Gebhard (6) has 
reviewed these papers by Motokawa, no 
attempt is made here to give a complete 
account of them. 

Of primary interest in Motokawa's 
work is the time course of the enhance- 
ment following lights of various colors. 
Figure 1 shows this time course for 
white, green, blue, and red lights. The 
ordinate is the zeta-value, Motokawa's 
measure of enhancement, where 

E—E, 
g = —— — X 100 
E 
In this expression E is the momentary 
excitability or reciprocal of electrical 
threshold at some particular time after 
the flash of light, and E, is the reciprocal 
of an electrical threshold determined by 
delivering test pulses during a portion 
of the experiment in which no light is 
present. The abscissa is the dela 
Le, the interval of time betw: 
cessation of the flash and the 
the electrical test pulse. In th 
ments, the duration of the light is ə 
sec. and the duration of the test pulse 
15 0.1 sec. Te will be noted that t rises 
steeply immediately following the flash 


een the 
onset of 
езе experi- 


y time, 
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Fic. 1. Curves obtained by Motokawa (16) for 
the electrical excitability of the human eye as 
functions of time after flashes of light. Two- 
second flashes of red, green, blue, and white 
light were used with a 2° ficld centrally fixated 
(upper graph) and in the periphery (lower 
graph). The ordinate is the zeta-value as de- 
scribed in the text. These results of. Motokawa 
are to be compared with Figures 6 and 9 of 
the present series of experiments, 


of light. Thereafter, each function 
reaches à maximum at a delay time that 
is characteristic of the color of the pre 
ceding light. Motokawa reports that 
these "crest times" are highly stable at 
approximately 1 sec. for red, 2 sec. for 
green, and 3 sec. for blue light. The ab- 
solute size of % is related to the intensity 
of the light and varies somewhat from 
one 5 to another, but the shapes of the 
functions are said to be highly reliable 
and the crest time for each wave length 
exhibits little variation from one exper 
ment to another. Hence Motokawa СО!” 
cludes that his method has revealed m 
specific temporal aftereffects of light ES 
the three. fundamental receptor system” 


| : со” 
for color in accordance with the tricl 
Matic theory, 
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Further support for the trichromatic theory is 
given by many later experiments reported by 
Motokawa. ‘The most fundamental of these is 
one in which spectral lights were used to pre- 
expose the eye. The results are shown in Fig. 2. 
Again the ordinate is ©, the measure of relative 
enhancement of electri threshold by the pre- 
exposure to light. This time, however, the let- 
ters М, G, and Ba iot used to represent red, 
green, and blue lights as they did in Fig. I. 
Instead, the letter XR is used to designate а func- 
tion relating enhancement, C, to the wave length 
of the light flash under conditions in which 
there is a delay time of 1 sec. between the flash 
and the test pulse. A similar function for a 
delay timc of 2 sec. is labeled С, and the third 
function, for a delay time of 3 sec., is labeled 
B. These particular delay times are the “crest 
times,” or delay time yielding maximum en- 
hancement, obtained for red, green, and blue 
pre-exposing light in the previous experiment. 
(See Fig. 1.) 

The curves are labeled К, С, and В in Fig. 2 
because Motokawa believes that these delay time 


0-5% 500 600 700 
Wave-Length ~ My 


а... ы 

(equal energy spectrum) 
Fic. 2, Curves obtained by Motokawa (16) for 
the electrical excitability of the human eye a 
functions of wavelength of preceding flash o 
light. Two-second flashes, 2° central field. B, 
curve obtained for a g sec. delay time after the 
flash. G, curve for a 2 sec. delay time. R, curve 
for a 1 sec. delay time. The lower graph con- 
tains the three “physiological sensation curves 
obtained by correcting the upper curves for 
Spectral energy distribution. 


curves represent the fundamental 
curves for the three basic color processes. The 
rationale for using delay times to single out the 
individual color response mechanisms has not 
been fully explicated in the papers by Moto- 
kawa. It is claimed, however, that curves such 
as those in Fig. 2 "can be regarded as an ex- 
pression of real physiological proce: in the 
retina obtained by direct observatior 
that they may be called "physiological s 
tion curves." Thus Motokawa distinguishes be- 
tween them and the "fundamental sensation 
curves" derived from color mixture and other 
psychophysical data by Maxwell, Abney, Konig, 
Hecht, Stiles, Wright, Walters, Pitt, and others 
(sce Wright, 25). 


response 


p 


The seemingly direct approach that 
Motokawa has used is based on certain 
assumptions whose validity may certainly 
be questioned. Implicit in the use of this 
method is the assumption that, at a de- 
lay time of 1 sec. after the flash, the 
electrical sensitivity of the eye is en- 
hanced primarily by the aftereffects of 
the light on the red response mechanism 
of the eye. It is on the basis of this as- 
sumption that the curve in Fig. 2 for a 
1-ѕес. delay time is designated as R, the 
curve for the red response mechanism. 
It is similarly assumed that for the 2-sec. 
curve, G, it is the green mechanism that 
is predominantly involved; and for the 
g-sec. curve, B, the blue mechanism is 
assumed to predominate. It is difficult to 
understand how these assumptions can 
be made, however, in view of the shapes 
of the curves in Fig. 1. These curves 
indeed exhibit maxima at approximately 
the times indicated for red, green, and 
blue light respectively. But they also 
indicate that (а) at 2 sec. the effect of 
red light is fully half as great as at 1 
sec.; (b) at 2 sec. the effect of blue light 
is almost as great as it is at its peak of 
g sec; and (c) green light is almost 
equally effective in producing enhance- 
ment of electrical sensitivity at delay 
times of 1, 2, and 3 sec. after the flash. 
In other words, the curves of Fig. r are 
not so sharply peaked at the times of 


4 L. A. RIGGS, JANET С. CORNSWEET, AND W. G. 


r, 2, and з sec. as 10 suggest that each 
of these particular times is likely to 
characterize exclusively the delay time of 
any one of the (presumptive) three funda- 
mental response mechanisms for color. 


Motokawa has taken some account of the above 
difficulty by assuming that the degree of en- 
hancement at a delay time of 3 sec., for example, 
is not exclusively determined by aftereffects oc- 
curring within the blue response mechanism. 
Enhancement is greatest, and therefore the 
threshold for electrical stimulation is lowest, 
with the blue mechanism at a 3-sec. delay time. 
But some degree of enhancement is thought to 
exist within the green mechanism and also the 
тей. Hence the concept of "multiple thresholds," 
which plays an important part in Motokawa's 
theory of enhancement. It is Motokawa's con- 
tention that the lowest, or "true" threshold is 
one whose value at 3 sec. delay time is fixed 
by the action of the blue response mechanism. 
Higher thresholds exist, however; and these 
"apparent" thresholds result from the lesser 


actions of the green and red mechanisms at this 
delay time. 


In summary, the work of Motokawa 
and his associates has offered a new ap- 
proach to the problems of human color 
vision. Our interest has centered on the 
"physiological sensation curves" that 
have emerged from the experiments in- 
volving direct electrical stimulation of 
the eye. These curves have been worked 
out on the basis of test pulses of direct 
current applied to the eye at fixed in- 
tervals following flashes of colored light. 
The electrical test procedure reveals that 
electrical sensitivity is enhanced by light 
and that the time course of the enhance. 
ment is specific to the wave length being 
used for the flash. For the extension of 
these basic experiments to the topics 
of fatigue, form perception, etc., the 
reader is referred to Gebhard’s compre- 


hensive summary (6) and the original 
works cited therein, 


THE PURPOSE or THE PRESEN 
INVESTIGATION 
Our main goal in undertaki 


r ng these 
experiments has been to furnish an in 


T 


LEWIS 
dependent check on the basic ен, 
length effects reported by Motokawa anc 
his collaborators. We have made no ae 
tempt to repeat the whole extensive 
series of experiments. Instead we have 
chosen to concentrate on the ‘baste 
phenomenon underlying much of this 
research, namely that of specific мр. 
length effects in the enhancement j 
electrical excitability following flashes 
of light. We have made every effort " 
observe wave-length effects under vary 
ing conditions of stimulus arca and in- 
tensity. р 
The method followed in these exper" 
ments was fundamentally that develope“ 
by Motokawa. However, we felt Mer 
pelled to introduce certain modification 


i { — [ the 
in the interests of standardization of i 
procedure. These modifications, " 

of this 


scribed more fully in later sections "hey 
report, are considered important. th aol 
include the following: (а) Provisions E 
defining the intensity of electrical tim б 
lation in units of current, independen 
of changes in skin resistance. (b) T 
use of equal step intervals in me du 
scending series of stimuli for all ше > 
old determinations. (с) Automatic ^ne 
trol of all time intervals used 1? 1565: 
presentation of light and current P Я 
(d) А prearranged protocol for ihe Ий 
entation of the stimuli, for m fof 
every judgment made by the S, ant rie’ 
the termination of the descending 55 ed: 
as the criterion for threshold is ent 
(е) Frequent determination of Tel" in 
electrical thresholds without light pon 
order to define the effects of light vipe 
the threshold. (f) Employment 9 ie any 
obtained data rather than discard? jn 
results on the basis of variab! á 
reference thresholds. qucd 
These modifications were ET "m 
With the expectation that they co 
serve to reduce the effects of 7 ene 
trolled variables in such expe? 
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For this reason it was anticipated that 
the effects of light itself would be more 
readily apparent їп the results. It is 
apparent, however, that the use of these 
modifications means that we have not 
repeated exactly the experiments of 
Motokawa. Thus any basic differences 
between his results and ours are per 
haps attributable in part to differences 
in procedure. Because of our concern 
with procedure we have made at the 
outset an extensive study of the varia- 
bility characteristics of the basic method 
used by Motokawa as compared with 
the conventional method of constant 
stimuli. 


APPARATUS AND PROCEDURE 


In all of the experiments to be те- 
ported here, the S was seated in a light- 
tight, electrically shielded room. ‘The 
experimenter was in an adjoining room. 
A thin partition separated the two rooms 
as indicated by the block diagram in 
Fig. 3. 

Arrangements for Stimulation of the 
Eye 


P system. The optical system is adapted 
Sent one described by Johnson (11) їп an 
Carlier report from this laboratory. It consists of 


= 000R 


LIGHT TRAP 


STIMULUS 
CONTROL 


PANEL 


CAM 
SWITCHE 


OPTICAL 


BITE. BOARD 


SHUTTER 


DISC GROUNDED SHIELD 


i Fic. 3. Block diagram of experimental arrange- 
ents. Subject's room on right, experimenter’s 
оп left, 


a tungsten source, collimating lenses, neutral 
density and color-selective filters, a rotating disc 
shutter, and a final lens that focuses the light 
on a spot within the pupil of the S's right eye. 
Auxiliary devices include a dull red fixation 
point and a diaphragm to limit the size of 
field. A recent attachment to the apparatus isa 
special eyepiece that may be inserted to extend 
the ficld diameter to 38°. This consists of a 
negative lens, placed between the eye and the 
last lens of the usual optical system, and a posi- 
tive lens of short focal length used next to the 
eye. The negative lens renders parallel the rays 
emerging from the regular optical system, and 
the positive lens focuses them on the pupil of 
the eye. A bite board of dental impression wax 
is used to position the head of the $ so that no 
part of the light is occluded by the edges of the 
pupil. This positioning is easily accomplished 
by an experienced $, since the rays emerging 
from the apparatus are brought to a focus that 
js smaller than the natural pupil of the eye. The 
S views the last lens of the system filled with 
light ("Maxwellian view"). The fixation point 
appears to be at the center of this lens. 

Calibrations of the intensities of the stimulat- 
ing lights were made by the procedure outlined 
in an earlier report (Riggs, Berry, and Wayner, 
29). By selecting appropriate filter combinations 
it was possible to present filtered light stimuli 
of various dominant wave lengths and of in- 
tensities such. that they were of equal effective- 
ness in stimulating the eye. This equality could 
be achieved on the basis of cither a photopic or 
a scotopic luminosity function. 


use of the Maxwellian view, 
ere not affected by the con- 
Johnson (11) has 
in specifying 


a 

4 Because of the 
the rays of light w 
traction of the natural pupil. 


discussed the resulting difficulty 
the photometric intensities of such stimulus 
fields. ш the present experiment, the small 


(2°8') field of white light reduced 3.7 log units 
by neutral density filters matches a comparison 
ficld of white light adjusted to 12.8 foot-lamberts 
and viewed with the natural pupil. Colored 
lights were provided by selective filters described 
by Riggs, Berry, and Wayner (23). On the basis 
of their calculations of the photopic equivalence 
of these stimuli we may state that each of the 
following combinations has the stimulating effect 
of a 12.8 ft..L. field: White, with neutral density 
filters of 3.7 log units density; red (C) with 2.8 
log units of neutral density filters; green (G) 
with 2.3 log units of neutral density filters; and 
blue-violet 76 with no neutral density filters. 
These stimuli are calculated to be roughly 
equivalent in their effectiveness for arousing the 
central cone receptor system of the retina. 
The insertion of the special eyepiece (together 
with the removal of an rture stop in the 
apparatus) has the effect of reducing the lumi- 
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Fic. 4. Diagram of the circuit used for elec- 
rical stimullation of the eye. B, go volt battery; 
LS, line switch; V, voltmeter; 5, S, cam Oper- 
ated switches; KS, experimenter's knife switch; 
Ry Rs resistance coils of potentiometer; Rs 
istance of 200,000 ohms; E, stimulating 


electrodes, 


Timing of the optical stimuli was provided 
by means of a rotating disc shutter that inter- 
cepted the beam at its narrowest focus. An open 
section of the disc permitted. an exposure of 
2 sec. during cach Cycle of approximately 15.9 
sec. The “wiping time" for opening or closing 
the shutter was :03 sec. 

Electrical system, A di 


agram of the apparatus 
for stimulating the eye 


With electric current is 
shown in Fig. 4. A D.C, voltage is supplied 
by the battery at B in the diagram. This is 
nominally а go-vo]t supply but its actual value is 
indicated by the voltmeter, V. This voltage is 
divided by the use of a potentiometer circuit 
containing two Cenco decade resistor units, 
R, and R, The dials for these units were 
manipulated in such a way that the sum of the 
resistances set into R, and R, was always equal 
t0 10,000 ohms, Thus, the voltage available for 
stimulating the eye was always proportional to 
the resistance set into R. Specifically, this volt- 
age, Vs, is given by the relation 


VR, 


10,000 
where I’ is the battery volt 


age indicated by the 
D.C. voltmeter, R 


2 Can be adjusted by fixed 


nance while incre: sing the size of the field in 
order to include more of the peripheral rod- 
receptive region of the retina. Now white light 
with 2.0 log units of neutral density is the 
equivalent of 11.3 ft-L. Again referring to the 
work of Riggs, Berry, and Wayner we find that 
there is an approximate scoto, 
white light with 2.6 log units of neutr 
filters; red (C) with no 
(G) with 1.5 log units; and blue 76 
filters, These stimuli are calcula 
equivalent in theiragffectiv, 'ousi: 


peripheral rod receptor system of the reti 
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steps of 1 ohm from о to 10,000 ohms, but m 
actual opcration Steps of 10 ohms or morc arc 
used over a range of 1,000 to 7000 ohms, 
The electrodes at E consist of two silver plates 
held in contact with the skin of the brow and 
check at regions as near the eye as convenient 
directly above and below the eye. The check 
electrode has a square contact surface of 30 X 30 
mm. It is grounded and serves as the cathode in 
clectric stimulation. The electrode on the brow 
is a rectangle measuring 32 mm, long by 25 mm. 
high. The electrodes are held against the skin 
with approximately constant pressure by the use 
of a headband and spring device. Electrode 
paste is used to ensure good electrical contact 
with the skin. The electrical resistance is of the 
order of 4,000 ohms across these electrodes. 
In series with the S is a fixed resistor, R,, of 
200,000 ohms. This is a precision resistor entirely 
enclosed within a metal shield. Two important 
functions are served by this arrangement. Fi st, 
it limits the current that can flow through the 5 
so that even a Charge of several hundred volts 
in the experimenter’s room would be harmless 
to the S. Second, it permits ап accurate state- 
ment of the current flowing through the 5 in 
terms of the setting of T. The stimulating 
current, 7,, in amperes is given by the relation 


V, 


Amal 
200,000 + R, 

where V, is the voltage supplied by the potenti- 
ometer as indicated above, and R, is the resist- 
ance of the S. It will be noted that, with Ra 
typically about 4,000 ohms, even a relatively 
large change in R, has very little effect on the 
value of 7,. It may therefore be said that, to an 
accuracy of one or two per cent, the stimulating 
current is proportional to the value of R, in the 
potentiometer, and its absolute value, 7,, may 
always be specified in terms of the above equa- 
tion, 

It has been found necessary to ground the 
stimulating circuit and place the 5 within an 
clectrically shielded room. The check electrode 
Was grounded, and the polarity of stimulation 
Was such that the forehead electrode was made 
positive by reference to it. 

Timing arrangements. The switches S, and. 
are microswitches actuated by a cam device 
mounted on the rotating disc shutter, ‘The rela- 


a 


* Motokawa has indicated that polarity has 


little effect on the degree of enhancement. We 
have confirmed this for the 0.1-sec, pulse йш 
tion used with the dark-adapted eye. Howarth # 
(10) experiments have shown that both on-and 
olf effects are found with this duration and that 
it is a Suitable duration because the strength- 


: t 
duration curve has reached a steady value а 
about ол sec. 
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tion between the open sector and the cam could 
be adjusted by reference to a radial scale, so that 
any desired delay time could be provided be- 
tween the end of the light flash and the onset 
of the D.C. pulse. The cam operates in such a 
Way that, just prior to the time at which a 
stimulus is to be given, S, is closed and then S; 
is closed. Then stimulation is begun when S, 
drops open, allowing current to flow through 
the $, and ол sec. later S, drops open, thus 
terminating the flow of current from the poten- 
tiometer, These switches remain open until 
nearly the time for the next pulse, when they 
are closed in the order S,— S, as before. While 
the full cycle of the shutter ¢ i 
is possible to provide clect 1 pulses separated 
by only half of this interval by inserting a 
second cam in a position diametrically opposite 
to the first. This is ordinarily done for speeding 
up the determination of electrical thresholds in 
"dark" series, i.e, when no light flashes are in- 
volved. Control experiments have shown that 
similar thresholds arc found with stimulus in- 
3 and 7.65 scc. 

‘The experimenter uses a noiseless knife switch, 
KS, to regulate the presence or absence of cur- 
rent, for those series in which the § must make 
a judgment of this kind. 

Controls. The timing and intensity of electri- 
cal stimulation were continuously monitored by 
the use of a cathode ray oscilloscope. In addition, 
photographic records were made from time to 
time in which both the flash of light and the 
square-wave electrical pulse were registered on 
Moving photographic paper. A Hathaway oscillo- 
Braph was used for this purpose. This gave 
assurance that the various stimuli were being 
delivered at the proper times with the desig- 
nated durations. It was found necessary to renew 
the microswitches from time to time. They 
typically delivered a clean square wave for 
Several months of operation, but then developed 
a slight chatter or raggedness of action. Although 
the total energy of the stimulus was scarcely 
altered by this degree of irregularity, switches 
Were discarded as soon as the raggedness became 
apparent on the oscillographic record. 


Psychophysical Methods 


Since one of the aims of the present 
Series of experiments was to verily cer- 
tain of the basic findings of Motokawa, 
It seemed important to use psychophysi- 
Cal procedures as similar as possible to 
those used in the original work. We have 
established a procedure for determining 
thresholds which incorporates the es- 
ential elements of the method which 
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he originated and uses in his work. 
Certain differences between this method, 
which will be called the limits-compari- 
son (LC) method, and that of Motokawa 
will be discussed after the LC method is 
described. 

The limits-comparison procedure. The 
LC method, as the name implies, is a 
two-stage procedure. It begins with a 
descending series of stimulus presenta- 
tions, as in the conventional method of 
limits. The series is begun with the 
presentation. of a stimulus which pro- 
duces a clearly visible phosphene. The 
exact intensity of this first stimulus is 
varied from one determination to the 
next. After a positive response, the in- 
tensity of the stimulus is decreased to a 
value 98 per cent of the preceding one. 
This procedure is continued, with each 
electrical stimulus being 98 per cent of 
the preceding one, as long as the S con- 
tinues to respond positively. When the 
S fails to see a phosphene, the same in- 
tensity is repeated on the next trial. If 
а phosphene is seen on the second pres- 
entation of the stimulus, the intensity 
is again lowered for the next trial. When 
the 5 responds negatively on two con- 
secutive presentations of the same in- 
tensity, the second stage of the pro- 
cedure is invoked. 

'The comparison stage of the method 
consists of presenting a pair of stimuli, 
one of which is a blank. The S reports 
which of the two is the test stimulus. 
The position of the blank, ie., first or 
second, is randomized and established 
before the determination is begun. The 
$ can respond by naming the position, 
first or second, at which he sees a phos- 


We wish to express our appreciation to Dr. 
Motokawa for sending us descriptions of his 
method and for visiting our laboratory with the 
purpose of acquainting us with the details ot 
his procedure. The most detailed published ac- 
counts of Motokawa's procedure for determining 
thresholds are found in references 20, 21, and 22. 
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phene, or, if he cannot discriminate, by 
saying "I don't know." The S is in- 
structed not to guess, and to make a 
judgment only if he is able to state 
which position is correct. After each 
judgment of position is made the ex- 
perimenter tells the S whether he is 
right or wrong. For those trials in which 
the $ is unable to state which position 
is correct, the experimenter does not in- 
form him of the correct position. 

The comparison procedure is started 
by presenting a pair of stimuli, one of 
Which is a blank and the other a test 
stimulus whose intensity the S has failed 
to see on two consecutive presentations 
in the method of limits. If the S cor- 
rectly identifies the position of the stimu- 
lus, the intensity is again decreased by 
2 per cent and another comparison pair 
is presented. If the S makes an error in 
naming the position of the phosphene, 
à second comparison at the same stimu- 
lus intensity is presented. If this com- 
parison is correctly judged, a third com- 
parison at the same intensity is pre- 
sented. If the third is also correctly 
judged, the intensity is decreased for the 
next comparison. Each time that the S 
reports that he cannot name the correct 
position, the same comparison pair is 
repeated. The criterion for threshold is 
three consecutive "I don't know" re- 
Sponses at a given intensity, or two in- 
correct lesponses out of three compari- 
Son pairs at a given intensity. These 
Criteria are not combined; that is, one 
or the other must be met. An $ can 
respond incorrectly once, say "I don't 
know” once or twice, and still continue 
to the next step by responding 


correctly 
on two further trials. The intensity of 
the criterion value is called threshold. 


A record is kept of all stimuli presented dur- 
ing a threshold determination together with the 
S's response to each. The number of minutes 
after the beginning of the session at which the 
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TABLE 1 


SAMPLE PROTOCOL ILLUSTRATING THE 
Limits-Comparison METHOD 


(Subject WGL 4/29/54— Dark Threshold) 


Limits St Comparison Stage 


R R. M Re- 
M ^s 5 "Оңа! Osl- . " 
Setting Response Setting Mod sponse 

4090 Yes 3200 2 S 

4010 Yes 3140 2 R 
3930 Yes 3080 2 

3850 Yes : 

3770 Yes 3020 2 5 

3690 Yes 3020 1 R 
3020 1 

3620 No ; 

3620 Yes 2960 1 R 

? 

3550 Yes 2000 2 R 
3480 Yes 2900 2 

3410 Yes d 

2840 2 P. 

3340 No 2840 1 ; 

3340 Yes 2840 I R 
2840 2 

3270 Yes Е 

2780 I 3 

3200 No 2780 1 › 

3200 No 2780 1 f 
2780 1 


э z A à s, gi V 
Threshold is at an Ry setting of 2780 ohms, Li 
Ing а current of 116 microamperes. 


T 
determination is begun and ended arc alo X 
dicated. Table 1 is a sample protocol for ities 
threshold determination, showing the intens 7, 
presented (їп resistance units) and the 
responses, 


d k esenta 
The interval separating the stimulus pres 


n in| 
tions is kept constant by means of the HE 
device described in the apparatus section. ding 
interval is fixed at one of two values, дереп ret 
upon whether the threshold is being determin, 
with or without a flash of light preceding a. 
electrical pulse. For reference or "dark" t^ sce» 
Old determinations the intervals is 7.65 the 
while for “light” threshold determinations in 
interval is 15.3 sec. This doubling of tn edule 
interval is the only difference in the Т ex 
for light and dark determinations. Contro pre 
periments have shown no difference in me bag 
cision or mean value of threshold determina phe 
in the dark with each of the time intervals. — 
$ is made aware that the stimulus is being Pu 
sented by the sound of the switches being ive? 
vated, as described earlier. No warning 18 бо" 
for the presentation of the light. The $, the 
ever, becomes familiar with the rhythm 9 arca 
Cycle and has no difficulty їп being pr od d 
for the light, or the phosphene. Once the t 
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old determination is begun, no break or rest 
period is used, Stimuli are presented continu- 
ously until the threshold is reached. On the rare 
occasions when the S is unprepared for the 
stimulus because of a wink, cough, etc, he is 
allowed to ask for a repetition of that stimulus. 
He is not allowed to ask for a repetition of the 
stimulus for the sole purpose of re-evaluating it. 

The time required to determine a threshold 
varies considerably from 5 to $ and from one 


determination to another, The experimenter 
determines the starting point arbitrarily, at- 
tempting to estimate this value such that ap- 
Бр eight to fifteen steps will be used 

the 


method before the S begins the 
5 зе. The initial presentation must 
: seen clearly by the S. If it is not, the deter- 
mination is started again at a higher intensity. 
Ordinarily, at least four or five intensities are 
correctly judged in the comparison procedure 
hefore reaching threshold, but there is great 
Variability in this. For a dark threshold, the 
total time taken va from three to twelve 
minutes, with an average of about five minutes. 
When light is presented, this time is about twice 
as long. 


limi 


Differences between the LC method 
and that of Motokawa. As stated earlier, 
ап attempt was made to make the LC 
Method as similar as possible to the 
method originated by Motokawa. The 
differences between these methods must 
be made explicit, however, so that the 
reader can consider these differences in 
evaluating the results. 


" "ив differences between the methods appear 
‘lifter, of major importance. The first of th à 
rences is in our use of equal log steps 
throughout all series of stimulus presentations. 
Motokawa’s method is to use large steps of ten 
Per cent or more in the beginning of a thresh- 
old determination (ie, during the limits stage) 
апа then to use small steps during the final 
Portion of the comparison stage. Around thresh- 
Ыш his steps are approximately one per cent 
differences,  Motokawa docs not establish the 
Steps that he will use before the determination 
15 begun, 
Our own method has been to use а logarithmic 
Series of predetermined stimulus intensities such 
that each successive step in the series is 2 per 
Cent lower than the preceding one, Our reasons 
for insisting upon this mode of presentation are 
the following: (a) Results by the method of 
Constant stimuli have shown that there is es- 
Sentially а Gaussian probability relationship be- 
“een the probability of secing а phosphene and 


the log of the stimulating current. It is therefore 
appropriate to use equal log steps as the basis for 
any descending scale of stimulus intensities. (b) 
The use of equal log steps achieves, in à sys: 
tematic way, Motokawa’s twin aims of avoiding 
the use of very lengthy series of stimuli and yet 
basing a threshold determination on steps of 
small linear magnitude. (c) The use of equal log 
steps has the further advantage that there is no 
massing of stimuli at any one level of intensity. 
The probability of finding a threshold at any 
given intensity level is raised by increasing the 
number of stimuli presented at that level. In 
other words, an equal-step method is a safeguard 
against experimenter bias in the determination 
of thresholds. This point is discussed more fully 
in the final section of this paper. 

The second major difference, related to the 
first, is in the rigidity of the presentation of 
and the criteria for threshold. Moto- 
kawa's observers are allowed to ask for the 
comparison procedure at any time during the 
limits stage at which they consider the detection 
of a phosphene to be difficult. They are also 
allowed to ask for a repetition of a comparison 
pair if they are not sure of a response, or if 
they are “confident” of a response which proves 
incorrect. Another feature of Motokawa’s method 
is that he continues to present comparison pairs 
after the $ is no longer discriminating. This is 
done because of the possible existence of multiple 
thresholds, considered important by Motokawa. 
The S of multiple thresholds will be treated later 
in this paper. The threshold criterion is not 
rigid in Motokawa's method, and а threshold 
is decided upon after this continuation of the 
comparison series has led to a number of 
failures to discriminate. Again in the interests 
of low variability and standardization of the 
in the LC method all stimulus pres- 
entations are determined by the S's response to 
the previous stimulus, and the rigid criterion 
outlined above is used for establishing the 


threshold. 


stimuli 


proced ure, 


Subjects 

The reliable observation of an electri- 
cally aroused phosphene is a task which 
requires rather extensive training. As a 
consequence, all of the experiments to 
be reported have used a small number 
of trained Ss, usually three or four Ss 
per experiment. Most of the Ss took part 
in several of the experiments. 

The Ss were laboratory personnel, in- 
cluding graduate students and two of 
the authors. All were tested by Ishihara 
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plates (5th ed.) and found to have nor- 
mal color vision. Those Ss who custom- 
arily wore glasses were allowed to wear 
them during the experiments. Most of 
the Ss were familiar with the general 
purpose of the experiments, but every 
effort was made to prevent them from 
responding in terms of extraneous cues, 
forced choices, or preconceived notions. 
In judging single stimuli they were in- 
structed to pay no attention to serial 
position but to call each as it appeared 
to them. In the case of paired stimula- 
tions they were asked to designate the 
one that appeared to be the stronger of 
the two, but were never forced to choose 
between two that appeared equal. 


All Ss were trained for from eight to ten 
one-hour sessions prior to taking part in an 
experiment. This training consisted of practice 
in observing and reporting phosphenes, in dis- 
tinguishing phosphenes from blank stimuli, in 
following the routine of the limits-comparison 
procedure, and in observing phosphenes after 
light flashes. Training was continued until all 
Ss arrived at a point where repeated thresholds 
under the "no-light" condition were reasonably 
stable. As training progressed the discrepancy be- 
tween thresholds determined under similar con- 
ditions decreased markedly. By the end of the 
training period the second of two consecutively 
determined thresholds seldom differed by more 
than 15 per cent from the first. Only one 5 
was not able to achieve this degree of stability 
and found it difficult to distinguish a test stimu- 


lus from a blank. He was not used in the ex- 
periments. 


EVALUATION Or THE LIMITS-COMPARISON 
PROCEDURE 

Experiment I. The Reliability of Thresh- 
old Determinations 

The LC procedure has been used in 
the majority of our experiments for the 
following reasons: (a) It is basically the 
method developed and used by Moto- 
kawa in obtaining the specific wave- 
length effects in which we are primarily 
interested. (b) The procedure is rela- 
tively fast. A simple threshold may typi- 
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cally be determined by a single L.C run 
of about five minutes’ duration. A com- 
parable determination by the constant 
stimulus method would take 15 to 20 
minutes, and a similar time would ре 
required for a typical set of a 
and descending series in the method о 
limits. The slower methods cannot well 
be used in sessions involving the ten or 
more threshold determinations that W€ 
have found necessary to use in the pres 
ent experiments, ВР 

Since the LC procedure is а relatively 
new one, there is little information as 
yet on its reliability except for the re- 
ports of Motokawa and his associates. 
The numerous papers by Motokawa О 
vey the impression of extraordinary ем 
liability in the obtained electrical thres 
olds. Specifically, there is the statemen 


А 3 бола 4 reler-. 
that successive determinations of a ТЄ 


ence threshold ordinarily agree within 8 
few percentage points; a discrepancy i 
large as 10 per cent is relatively Bp 
mon for these reference thresholds — 
at various times during a given um 
mental session. A discrepancy 45 pus 
as this is taken by Motokawa to pt 
that something is wrong with the rest 1 
of that session, and they are discard 

The high reliabilities reported а 
Motokawa for his limits-com parison Suis 
are certainly not typical of abso UR 
threshold determinations in eet 
Agreement within 0.1 log unit (ОГ A re 
20 per cent) is ordinarily diu n 
quite satisfactory for individual: such 
minations of absolute thresholds 1n ki 
an experiment as the course of С 
adaptation. Howarth (10), Clause" oy 
and Gebhard (6) have stated that 
were unable to achieve such а p 
gree of reliability. It therefore els: 
desirable to make a detailed comp? еШ“ 
of the reliabilities of the new LC "f of 
od and the conventional meth? 


—— 
eS Se و وسا‎ 
E 2 — 
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иар (CS). The design of the 
Шоп amet 15 - h that each experi- 
ede provides data from both 
fec " the same Ss under the same 
бын cad conditions. It must be 
ime Asinio that the LC method 
йде * is not precisely the method 
edd; hc Motokawa. It has been 
ERE of dd described above in the in- 
St E greater objectivity. 
T umm in N | we have performed. a 
half of i er experiment mM which 
mining al al time was spent in deter- 
Ше у: -— ute electrical thresholds by 
the ale ad and half ofthe ume by 
шры. P ie cs method. A sufficient 
Dur auo ешп» were obtained so 
regard to yy could be drawn with 
Bene ne variability of threshold 
methods s <n by each of the two 
of the "б the absolute magnitudes 
Сл Г 10145 obtained by each. Due 
CS iM taken of the fact that the 
for thresh S a much longer time 
tempt S ч determinations. No at 
lowing "lae to find thresholds fol- 
the рет of light in this portion of 
reference res because it was felt that 
basic to baeo in the dark were 
Мања the findings reported by 
More Mv апа that the method could 
Use of si eniently be evaluated by the 
COP IT aerei 
Mesa rysical procedure. Experiment 
above je a LG method, as described 
of ч A the conventional method 
used as Mo EE The cs method was 
Old for ows to determine the thresh- 
Берн э Pang induced "phos 
rated Був even stimulus values, sepa- 
intengit qual log steps, were used. Each 
Arrangem was judged 20 times. A random 
сеа lie of the seven intensities was 
every 7.6 at the rate of one stimulation 
entation 5 sec. After a total of то pres 
5, Te at. each. intensity) 4. ОН 
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minute rest period was given. The order 


was then repeated in the opposite direc- 
tion, making a total of 140 presentations. 


Six blank stimuli were inserted into the 


order at random interva 


15 and the $ 


was informed of the blank after his 
judgment was given. This served to warn 
the S against false positive judgments. 
The S's task was to make a judgment 
on the presence or absence of a phos- 


phene at 
“yes” or “no 
sponse was recor! 


each stimulation. He called 
" after each and his re- 
ded. No comments were 


made by the experimenter except follow- 


ing the response to 
when the $ w 


a blank stimulus, 
as told that the stimulus 


had been blank. If the S missed a presen- 
tation by being unprepared for it, he 
was allowed to omit it and the stimulus 


was repeated at the еп 
The percentage О 
each stimulus intensi 


d of the series. 
f "yes" judgments at 
ity was calculated 


and plotted on a probability plot against 


the log О 
straight line м 
points, and t 
the intersection O 
per cent level on t 
No threshold was c 
the data unless the 
judgments covered 


from 25 рег cent to 75 Pe 


intensities used. 

Plan of each Session. 
Experiment I consis 
stimulus ( 
each taking 
two go-minute bl 


minations by t 
method. The LC blocks 
exactly 20 minutes 
time required to 
varies. In this case, 
were obtained within a 
roximately 20 ™ 


f the stimulating current. A 
as fitted by eye to the 
he threshold was taken as 
f this line and the 50 
he probability scale. 
onsidered as part of 
percentage of “уез; 

at least the range 
r cent for the 


Each session of 
ted of two constant 
CS) threshold determinations, 
оо minutes to obtain, and 
ocks of threshold deter- 
he limits-comparison (LC) 
could not be 
in duration, since the 
obtain а threshold 
several thresholds 
period of ap- 
inutes. The number 


of thresholds determined in 20 minutes 


say, Research 
xG COLLEGE 
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varied from two to five. A one-minute 
rest followed the first and third 20- 
minute periods in the session; and a 
three-minute rest followed the second 
period. Preceding the first period there 
was a 15-minute period of dark adapta- 
tion. During the latter part of this dark- 
adaptation period one threshold was 
determined by the LC method. This 
threshold was not used as part of the 
data, but served as a "warm-up" for the 
S. It also gave the experimenter an in- 
dication of the level of sensitivity for 
that day, allowing him to select appro- 
priate starting points for the later LC 
determinations. 

Experimental design. Experiment I 
consisted of 16 of the sessions just de- 
scribed. These were held approximately 
three times a week and no two were 
ever held on a single day. The two 
methods of threshold determinations, 
LC and GS, were presented in an ABBA 
order on half the sessions and a BAAB 
order on the other half. The presenta- 
tion of the two orders was balanced. 

The same seven intensity values were 
used for a given S for each of the 32 
thresholds determined by the CS method. 
The values were selected on the basis 
of preliminary work with each $, and 
were not changed after the experiment 
was under way. As a result, there were 
a few instances in which no threshold 
could be determined for a given period 
because of failure to satisfy the condi- 


tion that the positive judgments cover 


at least the range from 25 per cent to 


75 per cent over the series of intensities 
used. 

Three Ss participated in this 
ment. All had had previous tr. 
observing phosphenes and in the routine 
of the LC method, as described in 
earlier section. In addition, all had h 
practice with the CS method, 


experi- 
aining in 


an 
ad 
during 
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which the range of intensities to be used 
in Experiment I was established. rhis 
range was .4o log unit (96 to 22 pa) 7 
E : * UO. ne or 
LAR, .476 log unit (63 to 188 ра) n 
SHH, and .476 log unit (84 to 251 ра) 
WGL. | vij 
Results. The data from Experimen : 
allow two types of evaluation of the ris 
method of determining phosphenc thres 
ci i n- 
olds. First, this €t 


method can be 


pared with a more conventional a 
chophysical procedure, the CS en t 
Second, the large number of threshok 
determinations by the LC method makes 
possible an internal analysis of the — 
acteristics of this new psychophysic# 
procedure. 0 ri- 
A comparison of the limits-com pa 
son and constant. stimulus methods. It 
has been noted that the L.C method + 
quires less time for the дештин 
of a single threshold than does the sd 
method. For the purpose of comparing 
the methods and balancing times, b 
equal amount of time during each ем 
sion was spent on cach method. phu: 


» a 

S T j 113 * 

there were four 20-minute periods i 
Г the 


1 for 
for 
five 


single session, two spent on each © 
two methods. Since a single per!oc 
the LC method allows time enough 
the determination of from two t9. ес 
thresholds, while only опе was ent 
in each period with the CS method, in 
means of the several LC thresholds v 
a given period are compared with out 
CS threshold value. However, an na 
standing advantage of the LC ER 
its quickness, and the method jons 
cally employed for single determina 

of threshold. It is therefore apt ў 
compare single LC thresholds WIH? pe- 
thresholds. In the results tabulate C5 
low the comparisons of the LC аА 
methods аге made both for the m yof 
threshold for a period and the first ^^ 

a period. 


ў 
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TABLE 2 


THRESHOLD VALUES OBTAINED BY THE Lixrrs-CoMrARISON PROCEDURE (LC) Ахр BY THE 
METHOD OF CONSTANT STIMULI (CS) DURING EXPERIMENT I 


(Values in the bod 


y of the table are resistance settings of Аз in ohms) 


Subject LAR 


LC LC LC LC 
Day CS cs |———— Day CS Gs 
ist | Mn rst | Mn Ist Mn ist Mn 

3700 | 3000 | 3030 | 3050 2 | 3440 | 3410 | 3410 | 3200 | 3500 | 4240 

3400 | 3490 | 2000 | 2900 3 | 3600 | 2780 2870 | 2530 | 2820 | 3870 

3700 | 3870 | 3200 3260 5 | 3250 | 2680 | 2750 | 2730 2700 | 3700 

3730 | 3730 | 3340 | 2090 5 | 3840 | 3340 | 2090 | 3080 | 2030 | 3050 

3320 — 2780 | 2700 о | 3760 | 3020 | 2790 | 2730 2730 | 3300 

3500 | 3630 | 3200 3170 12 | 3030 | 3200 | 3170 2780 | 2700 | 3440 

3270 | 3410 | 2000 | 3030 14 | 3400 | 355° 3680 | 2000 | 3090 | 3620 

3140 | 3300 2780 | 2730 15 | 3570 | 3340 3150 | 2960 | 3020 3670 

3136 | 3001 3610 | 310$ | 3101 2864 | 2047 | 3607 


3402 | 3644 


Subject SHH 


EG F LC i6 
Da CS CS CS 
Ist Mn ist Mn ist Mn 
2 2330 ү a 2740 | 2780 | 2740 3020 | 2060 | 3020 
3 2330 2400 2 2360 | 2330 | 2240 2430 | 2100 2660 
5 2280 | 2100 | 2 2600 | 2630 | 2650 2680 | 2660 | 2бос 
8 2030 | 2280 | 2 2580 | 2630 | 2030 3080 | 3100 | — 
9 2330 | 2430 | 2 2300 | 2530 2360 | 2480 | 2430 | 2470 
uic 2580 | 2480 | 2 2500 | 2430 2380 | 2530 2460 | 2410 
ти: 22бо 2350 | 3270 | 3200 2780 | 2780 2670 
23 2530 | 2 258о | 2680 2880 | 3270 | 2920 | 2370 
Mean 2274 2250 2 2512 | 2660 2672 | 2784 2687 | 2боо 
Subject WGL 
"XN LC LC іс | a 
ا‎ S CS t — Bay 6S ——T-—— x cem 
Ist Mn Ist Mn Ist Mn | 15% Мп b. 


3340 | 3240 | 3100 | 3599 3140 | 3380 
2680 | 2960 | 2880 | 2660 2780 | 2860 
2630 | 2560 | — — | 3770 | 2880 
3020 | 2900 | 2960 | 3340 2630 | 2460 
3040 | 3040 | 2900 2990 
2870 | 3370 | 290° 2780 
2840 | 2840 2680 
2830 | 2730 | 2000 


3410 | 3410 
2780 | 2840 | 3070 


1 | 3130 | 3770 | 3270 2580 | 2430 | 2060 
4 | 2600 | 3080 2760 | 3410 | 3130 | 2750 
6 | 2980 | 3080 | 2040 2580 | 2510 | 2900 
7 | 3120 2840 | 3200 | 2580 | 2790 | 3240 
то | 3340 | 3410 | 3840 3480 | 3430 | 3370 
тт | 3310 | 3270 | 3340 3690 | 3450 | 3570 
13 2530 | 2320 | 2110 | 2000 | 3070 
16 


3140 
2840 | 2820 | 2930 


3100 | 3200 | 2770 


3270 | 2960 | 3010 


—— 
Day 
2 
3 
5 
8 
9 e 
12 3550 | 3370 
14 
15 
اس‎ 


2961 | 2866 


Меап 3085 | 3030 | 2000 | 3083 


Absolute thresholds obtained by the 
two methods. Table 2 presents the CS 
threshold, the mean of the LC thresh- 
Olds, and the first LC threshold. for 
ach of the four periods of each of the 
is sessions for each of the three Ss." 
‘gure 5 shows the means of these val- 


3106 | 3147 | 3055 | 2909 2831 | 3099 


ues. It will be noted that each value 


shown in Fig. 5 is the mean of eight 


"The missing values for CS thresholds result 
from a failure to meet the criterion of basing a 
threshold on percentages of positive judgments 


running from 25 per cent or less to 75 per cent 


or more. 
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1 2 3 m 
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Fic. 5. Mean thresholds for the elicitation of 
phosphenes in the dark-adapted eye. Threshold 
values for the first LC, mean LC, and CS deter- 
minations in the four periods of Experiment I. 


The individual thresholds and their means are 
tabulated in Table 2. 


sessions. The CS method was used eight 
times in each period in the 16 sessions, 
and the same is true for the LC method. 
Thus, in the bar diagrams of Ex Б, 
Periods 1 and 4 for a given method rep- 
resent the same eight sessions of the 
experiment, while Periods 2 and 3 repre- 
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TABLE 3 
OVER-ALL MEAN THRESHOLD VALUES 


riods € ined 
(Values of Rz in ohms for all per iods combi 
for each method) 


‘Subject CS LC(mean) Т.С) 
SHH 2531 2574 1300 
WGL 3070 2940 ped 
LAR 3010 3000 Я 


sent the other eight sessions. 
It can be seen from Fig. 5 
is no consistent tendency for pde 
old to increase as the session progres n- 
for either method. It must be ear ۴ 
bered that the Ss were dark-adapted : 
15 minutes prior to the first period a $ 
session. Hence the rapid — di 
threshold that are found during = i 
adaptation (Achelis and Миа а 
Lewis, 12) are not of any conseqt 
here. Р: 
A second finding evident from d 
is the similarity of absolute thresh E 
values for the two methods as ape 
by two of the Ss, SHH and ЖО, her 
other $, LAR, had a consistently h!8 ‹ 
threshold when measured by me êê 
method. The over-all means for the t 
measures are presented in Table 2 by 
Variability of thresholds а. de 
the two methods. Experiment I ез ай 
signed so that the two methods es 3 
CS) could be compared with. met. 
their within-session variability. 1s on 
each session consisted of two perio jia 
each of the methods. Further, fo! usel 
of the sessions a given method e 
to determine thresholds in 2 
time periods (Periods 2 and 3) ? й used 
the other half the same method e X 
in separated time periods (1 bility 0 
Hence, the within-session varia each 
the methods can be compared for da 
set of time conditions. The id ete 
variability of electrical threshol ants 2 
minations is large, and experime 


that there 
the thresh- 


in 
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this phenomenon are best designed so 
that thresholds taken on different days 
are not directly compared. Therefore, 
this type of variability will not be con- 
sidered here. 
{ The basic measure that will be con- 
sidered here is the average within-ses- 
ston change for each method. The meas- 
ure is obtained by computing the differ- 
ence between the two thresholds deter- 
mined by the CS method, for example, 
in a given session. Since the methods 
were presented in an ABBA order on 
half the sessions and a BAAB order for 
the other half, there will be, for each S, 
eight differences between Periods 2 and 
3 for the CS method and eight differ- 
ences between Periods 4 and 1. The 
mean of these eight differences gives a 
measure of the average change, or with- 
In-session variability for that method. 
An example of the computation of this 
average change is given in Table 4. 
ihe differences in Table 4 were ob- 
пе һу subtracting the threshold ob- 
С ın Period 1 from the threshold 
ымлый im Period 4 for each of the 
eck in which the CS method ap- 
ared in these periods. Reference to 
Table 2 in which these thresholds are 


TABLE 4 


SAMPLE CoMPUTATION OF THE AVERAGE 
CHANGE IN THRESHOLD VALUES 


(Subject WGL-CS Method; Period 4—Period 1) 


= Day Е" 
І d da 
1 + бо 
4 — 80 
: +120 
a + 3o 
19 +260 
13 “so 
16 але 
Mean (algebraic) ee 
ariance pret 
Mean (absolute) = 


presented may clarify the procedure. 
The mean of these values is obtained, 
both with and without regard to the 
sign of the difference. A positive differ- 
ence indicates that the threshold was 
higher, i.e., less sensitive in the later 
period. The algebraic mean is used as 
the basis for the calculation of the vari- 
ance of the measure. 

The above tabulation was followed for 
both the CS and the LC methods. For 
the LC method, it was done twice, once 
for the mean LC of the period and 
once for the first LC of the period. It 
was done for both the adjacent periods 
(5-2) and separated periods (4-1). Two 
main comparisons emerge from this tab- 
ulation. First, the absolute size of these 
within-session differences is of interest. 
The size typical of these differences is 
an indication of the variability of thresh- 
olds determined by each method. Sec- 
ond, the algebraic mean of these differ- 
ences is also important, since it provides 
for the assessment of trends through the 
session. Positive mean differences would 
decrease in sensitivity as 


indicate a 
The algebraic 


the session progresses. 
mean is also the proper basis for a vari- 
ance measure. Knowing the variance, 
one can make a statement of the limits 
of confidence of a given difference. 
Tables 5 and 6 present the results of 
this analysis of the two methods. In 
Table 5 the algebraic and absolute mean 
changes are presented, together with the 
over-all mean threshold values for each 
S. The mean thresholds, combining all 
are presented in order to give 
he magnitude of the differ- 
ted. Table 6 presents F 
he variances of the 


periods, 
meaning to t 
ences presen 
ratios, comparing t] 
methods. 

A consideration of Table 5 shows that 


the CS method has generally given more 
reliable thresholds than has the LC 
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TABLE 5 - 
1ТЕ кт z 5 TAINED IN 2 
AVERAGE CHANGES (OHMS, ALGEBRAIC AND ARSOLUTE) OF ‘THRESHOLDS OBTA 


SINGLE SESSION OF EXPERIMENT I nv A SINGLE METHOD 


LAR SHH bo ME 
p ү, Е wh. 
Absol.  Algeb. Absol. — Algeb. Absol. Mie. — 
d 272.80 — +02.80 

р. 127.14 127.14 131.43 +131.43 з eso 

Period 4-1 350.00 +87.50 182.80 +97.14 2 

- 070 a 

Mean Threshold 3610 2531 Е 3 
LC Method (mean) _ 

Period 3-2 176.25 —153.57 132.50 +15.00 

Period 4-1 323.75 231.25 310.00 4257.50 

Mean Threshold 3000 2574 | 
LC Method (rst) 8 

Period 3-2 313.75 —301.25 258.75 +123.75 

Period 4-1 395.00 -+220.00 403.75 +286.25 

Mean Threshold 3020 


method. This conclusion is based on the 
average discrepancy between thresholds 
determined in the two periods of each 
session devoted to a given method. The 
average discrepancy for the CS method 
is lower than that obtained with the LC 
method even when due allowance is 
made for the fact that individual LC 
runs take less time than the CS deter- 
minations. The magnitude of the dis- 
crepancy relative to the size of threshold 
is in every case less than 10 per cent for 
the CS method, less than 15 per cent for 
the LC mean, and less than 20 per cent 
for the first LC. The average discrep: 
between the adjacent periods, three 
two, is consistently smaller than tha 


ancy 
and 


ot 
Periods 4-1 for LAR and SHH, but pn 
so for WGL. These 3-2 period UE 
are of particular interest, since ici die 
dicate the variability of thresholds mi: 
termined closely in time, as in the Ron 
cal experimental situation. Since jetet 
are generally smaller than those йы 
mined far apart in time, it is conc 1 be 
that reference determinations shoul 
repeated frequently. 1 
The algebraic differences, also $ 


y : si 
in Table 5, are not consistently po 


1 own 
tive 


Eu 
Ре - И dur! 

appear to exhibit a rising — 
the session. One of the Ss, SH - 
ard a її 

Show some tendency toward a 


t 
Е ws 
tof threshold. Another $, WGL, sho 
TABLE 6 
F Ratios or VARIANCE ОЕ AVERAGE CHANGES — ge 
LAR SHH E WT 
> xpo 
LCs/CS LC,/CS LOw/C8 LCO,/CS LC,/C8 LCi 
Рено дна 3-404. 2.85 2.559, 
à -853 " . 
C-—- К: BE E O 
СТА 
** SÎ 


T In this case the CS variance w; 


as greater t' in the LC variance, 
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opposite trend as indicated by negative 
values in all but one of his discrepancies. 

The F ratios shown in Table 6 point 
out the fact that in all but one case 
the variance of the LC method changes 
was greater than that of the CS. Only 
for LAR, 4-1 period difference, was the 
CS more variable than the LC mean but 
not more variable than the LC first. 
None of these ratios for the 3-2 period 
difference is significant. For the Period 
4-1 difference, they are significant for 
WGL, again indicating that frequent 
determinations of a reference threshold 
are desirable. Variances combining the 
4-1 and the 3-2 periods were also com- 
puted for both methods. Here the LC 
difference variance was always greater 
than the CS. None of these was signifi- 
cantly greater when the mean LC was 
used, but for two Ss, WGL and SHH, 
the LC first/CS ratio was significant. 
When the variance of the first LC is com- 
pared to that of the mean LC, no signifi- 
cant differences are found for any S. 
In all but one case, however, the first LC 
variance was greater than the mean LC. 


Characteristics of the Limits-Comparison 
Procedure 


Variability. The large number of rep- 
etitions of the limits-comparison method 
(two to five in each 20-minute period) 
makes it possible to compute the vari- 
ability of thresholds determined by this 
method, Since the method involves pre- 
senting equal log steps, each represent- 
ing approximately a 2 per cent decre- 
ment in stimulus intensity, the differ- 
ence between any two thresholds may be 
expressed as the number of these steps 
separating the thresholds. It is conven- 
lent to make use of these step units for 
the purpose of setting up a distribution 


of differences between adjacent thresh- 
Olds, 


The details of the procedure which 
has been used to get at the question of 
variability are as follows. In each period 
there are several thresholds obtained by 
the LC method. The differences between 
successive thresholds within each period 
are found. These differences between the 
first and second, second and third, etc., 
are tallied for each S. The result is a 
distribution of differences, the mean of 
which represents the mean discrepancy 
between one threshold value and the 
next. The standard deviation of this dis- 
tribution can be used to estimate fidu- 
cial limits of this discrepancy. As in the 
earlier computation of differences, this 
mean difference has been computed al- 
gebraically, to assess trends and to com- 
pute the variance, and absolutely, to es- 
timate the average size of the discrep- 
ancy independent of direction. 

'Тһе results of this analysis are seen 
in Table 7. The discrepancies in this 
table are expressed in step units. 

A positive value in the algebraic mean 
data of Table 7 indicates that the second 
threshold was higher, i.e., less sensitive, 
than the first. The values labeled transi- 
tion threshold will be considered in the 
next section. It may be noted that for 


TABLE 7 
AVERAGE ADJACENT THRESHOLD CHANGE 


(in 2 per cent step units) 
LC Method 


Abso- Alge- 
lute braic № SD 
Mean Mean 
LAR 
LC Threshold 4:77 —-46 71 5.84 
Transition Threshold 3.61 —.06 71 4.49 
SHH 
LC Threshold 3.41 —.03 59 5-11 
Transition Threshold 3.95 +-15 59 4.96 
WGL 
LC Threshold g.to —1.01 69 6.53 
Transition Threshold 5.19 —1-30 69 8.15 
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each S the LC threshold difference is 
negative, but not large. From the infor- 
mation in Table 7, it can be predicted 
that the second of two adjacent thresh- 
olds will fall, 68 per cent of the time, 
between the limits of 6.30 steps below 
the first to 5.38 steps above the first 
for LAR, 5.14 steps below to 5.08 steps 
above for SHH, and 7.54 steps below 
to 5.52 steps above for WGL. The above 
values are obtained by taking one stand- 
ard deviation above and below the mean 
discrepancy. Since each step represents 
approximately a 2 per cent change, the 
percentage discrepancy expected can be 
computed approximately by multiplying 
the above values by two. 

Limits method vs. LC method. Since 
the first stage of the LC method is the 
descending phase of the conventional 
method of limits, a question arises as to 
whether the reliability of the method 
is increased by adding the comparison 
stage of the method. Motokawa reports 
that the method of limits by itself, with- 
out a comparison phase, is not suffi- 
ciently sensitive to be applicable to the 
study of electrical thresholds, 

Although the present experiment did 
not include a direct comparison of the 
LC method with the method of limits, 
it is possible to make a comparison be- 
tween threshold values Obtained with 
the LC method and the transition inten- 
SIty, Le., the value at Which the limits 
‚| =н 
ments of “No.” It fend ber us 
that this value, which ma оа e 
"transition threshold," i V о 
equivalent to а threshold ol s n 

tained by the 


usual method of limits, since the $ was 
aware that the second à 
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which would be expected from the ша 
od of limits. Clausen (5) has шн 
the comparison method with the met ‘tel 
of limits, using both ascending vi is 
scending series for the latter. W ts y^ 
not clear that he made any allowance 
the differences in time taken by the pe. 
methods, he concludes that the Hee 
method yields threshold sida ШЕ v 
but less variable than those pur 
the comparison procedure. It 15 € К 
that the transition threshold m 
ways be higher than the LC пиа 
However, the variability of the - 
tion threshold can be compared je 
that of the LC threshold. The procedi 
followed to obtain the adjacent O 
old discrepancies for the ppt the 
thresholds is exactly as described е de 
LC method thresholds. Table 7 ine 
these values. It will be noted mie 
changes obtained when the ae 
threshold is used are very simi’ 


Я od, а! 
those obtained by the LC metho any 


tion 
to 


: -urs in 
that no consistent trend occurs 
of the three Ss. 


contribute to the reliability of 
olds obtained by the limits phase d 
The LC method was nevertheles? out 
throughout the major portion aso" 
experiments for the following ? foto 
(а) It is the basic method used PY rect 
kawa to study specific waveleng" Jd 
(b) it yields thresholds that aT, . , цо 


Maintaining the $'s interest Pus 
ing him of the correctness ОГ ats- 
ness of his comparison judgme™ " qe 

Characteristics of the consta | 
lus method. The data obtaine abs 
CS method were typical © 
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threshold data in general. The results 
of each threshold determination were 
plotted on probability paper, with per- 
centage of “yes” responses on the proba- 
bility ordinate and log stimulating cur- 
rent in microamperes on the abscissa. The 
resulting points showed the usual degree 
of conformity to a linear relationship. A 
straight line was fitted by eye to the 
points of each graph. There were 32 
graphs for each S, two on each of the 
16 sessions. 

One index of the precision of the judg- 
ments is given by the ratio between the 
intensity that gives 75 per cent judg- 
ments of "yes" and the intensity that 
gives 25 per cent judgments of "yes." 
The mean value of this ratio is 1-42 
(152 log unit) for S LAR, 1-75 (.244 log 
unit) for $ WGL, and 1.53 (485 log 
unit) for $ SHH. The standard devia- 
tions of the distributions of these values, 
expressed in log units, are 0.027, 0.058, 
and 0.049 respectively. 

During the course of each threshold 
determination six blank stimuli were 
presented. That means that a total of 
192 blanks were presented during the 32 
determinations for each 5. The numbers 
of false positive responses obtained with 
these stimuli are as follows: For S LAR, 
one (0.5 per cent); for WGL, 28 (14 per 
cent); and for SHH, 20 (10 per cent). 


Conclusions with Regard to the Limits- 


Comparison Procedure 

ative conclusion from the re- 
sults of Experiment I is that the limits- 
comparison method has not proven to 
be more reliable than the conventional 
method of constant stimuli in the de- 
termination of thresholds for the ap- 
pearance of phosphenes in response to 
electrical stimulation. Furthermore, the 
over-all precision that we have found 
for the electrical thresholds is not 50 


А conserv 
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high in any of these experiments as in 
the ones reported by Motokawa. In the 
final “Discussion” section of this paper 
we have attributed the phenomenally low 
variability in Motokawa’s data to the use 
of psychophysical procedures that serve 
to curtail the range of intensities within 
which a given threshold is likely to be 
found. 


APPLICATION OF THE Limrrs-COMPARISON 
PROCEDURE TO THE STUDY OF 
ENHANCEMENT 


General. Procedure 

Electrical stimulation experiments are 
of interest chiefly in the possible inter- 
actions between electrical and photic 
stimuli. Motokawa has reported that the 
aftereffect of a flash of light is nearly 
always to enhance the sensitivity of the 
eye to electrical stimulation. The re- 
mainder of the present experiments con- 
sists mainly of determining the amount 
and time course of this enhancement 
[following single flashes of light. The 
procedure is first to find the basic sensi- 
tivity of the eye by determining the ab- 
solute threshold for electrical stimula- 
tion in the dark. Then successive flashes 
of light are presented, and the corre- 
sponding threshold is determined for 
electrical stimuli presented at a given 
delay time after each flash. The degree 
of enhancement is then expressed as the 
ratio of electrical sensitivities determined 
with and without the flash of light. This 
has been done for a number of different 
experimental conditions with regard to 
delay time, intensity of light, area of 
light, wave length of light, and certain 
procedural variations. The basic proce- 
dure is described below. In all the experi- 
ments dealing with the effects of light 
upon sensitivity to electrical stimula- 
tion, the LC method has been employed. 

A single experimental session. The 


20 


experimental session » begun ue ge 
positioning of the S's biting board so 
that the light enters the pupil of the 
right eye. This is accomplished by turn- 
ing on the light and allowing the 5 to 
make adjustments in the position of the 
biting board. The $ learns to do this by 
looking successively up, down, to the 
right, and to the left, meanwhile ad- 
justing the biting board so that the 
light disappears at about the same de- 
gree of rotation for each quadrant. After 
this is accomplished, the light is turned 
off and the S is dark-adapted for зо 
minutes. During the latter part of this 
dark-adaptation period one threshold is 
determined by the LC method, under 
the no-light condition. This threshold 
is not used as a part of the data but 
serves merely as a practice determina- 
tion. 

After the S has been in the dark for 
approximately 20 minutes, the first dark, 
or reference, threshold is determined. 
Following this, a "light threshold," i.e., 
an electrical stimulation threshold with 
light preceding each electrical stimula- 
tion, is determined. Next, another dark 
threshold is determined. This alterna- 
tion of dark and light thresholds is con- 
tinued throughout the session. The first 
threshold and the final threshold of every 
session are dark thresholds. Hence each 
light threshold may always be compared 
with the mean of two dark thresholds, 
le. those immediately preceding and 
following it. А one-minute rest period 
separates the succeeding threshold de- 
terminations, and no rest periods are al- 
lowed during any single determination. 
As previously described, intervals of 7.65 


® Previous experiments (Achelis and Merku- 
low, 1; Lewis, 12) have shown that the electrical 
threshold does not follow a Course simil 

the light threshold durin pu 


the hr 8 dark adaptation, Tt j 
fairly stabilized at 20 minutes, Р NUS 
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sec. separate stimuli in the dark condi- 
tion and 15.3 sec. in the light. . 

The time required for a single session 
varies considerably, depending upon the 
number of light thresholds determined 
and the time required for each thresh- 
old. In general, the sessions were de- 
signed to last one to two hours, includ- 
ing the period of dark adaptation. Oc- 
casionally longer sessions were under- 
taken, in order to get a large number 
of points in one session. No single 
sion exceeded two and one-half hours 
in duration. | 

Computation of the effects of light. 
The effect of a light flash on electric? 
sensitivity will be designated in the pres 
ent experiments in terms of an enhance: 
ment value e. 
as follows: 


ses- 


à : enei 
Enhancement is definet 


or, more simply, 

= = log Г, — log I = 
where J, is the mean of the two "дате 
thresholds (i.e., the electrical thresholds 
determined immediately preceding ant 
following the determination of ВЫ 
"light" threshold) and 7, is the “light 
threshold itself. All values of Z аге 2" 
terms of electric current in micron 
peres. It should be stressed that € is € 
culated with reference to the dark um 
that surround the particular light thres | 
old for which the value of e 35 pd 
tained. In this way, the effects of uit 
in electrical threshold which might де. 
during the session as a result of fatig". 


Меш 2 ation © 
etc., are minimized in the calculatio 
€. 

An 
Differences from Motokawa’s procedu! edu? 
Important modification of Motokawa's pxo лс 
Which has been introduced here is the P s e 
of repeating dark, or reference, thresho e the 
fore and after each light determination. pe cal- 
use of the mean of these two as a basis ‘pegin’ 
culating the effects of the light. Motokawa ® 
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each session with a dark determination, and oc- 
casionally repeats this throughout the s ssion, 
but apparently does not do so systematically. 
He uses the original value for calculating effects 
of light, and the repetitions serve only as a 
check on the stability of the 5. If a later dark 
threshold dillers [rom the original by more than 
ı0 per cent, Motokawa discards the data for 
that session, In the present experiments, how- 
ever, no data are. discarded, and a mean of the 
two dark thresholds is taken ardless of the 
magnitude of the discrepancy between them. 
Instead of Motokawa's C, as an expression of 
the enhancing effect of light. £ values are used 
in the present experiments. ¢ has the advantage 
of being expressed in logarithmic units as 
deemed appropriate in these experimer It 
carries with it the property that equal per- 
centages of enhancement OF depression of 
electrical. excitability are represented by equal 
positive or negative values of e. In other words 
a depression of the threshold of six step units 
or an enhancement of six step units has the 
same absolute value on а log scale when ex- 
pressed as g, but different values when expressed 
аз C. This modification of Motokawa's procedure 
is not in the same category those discussed 
earlier, however, since it involves no change in 
experimental procedure, but merely а slightly 
different way of expressing the data. It may be 
noted that É and ¢ cach have a value of zero 
when no enhancement is present, and that ğ and 
e have similar positive and negative values for 
all small amounts of enhancement or depression. 


Experiment II. Enhancement Following 
Flashes of Light From a Small Central 
Field 

A conclusion of major importance in 
Motokawa’s work is that the course of 
enhancement following a flash of light 
reaches a peak at a certain delay time. 
The delay time for this peak is called 
the crest time, and its value is said to 
depend chiefly on the color of the pre- 
ceding flash of light. Experiment II was 
designed to test the dependence of crest 
time on color in as direct a fashion as 
possible. 

Procedure. Since there was not enough 
time in any one experimental session to 
vary both color and delay time, à single 
color was used on any given day. En- 
hancement values were determined for 
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this color at the three delay times (1, 2, 
and g sec.) designated by Motokawa as 
the crest times for red, green, and blue 
lights respectively. The results were then 
evaluated simply by observing the degree 
to which the use of a red light, for ex- 
ample, resulted in a greater enhancement 
value at one second than at two or three 
seconds when these three enhancement 
values were found within a given experi- 
ment session. The design of the experi- 
ment was balanced in such a way as to 
minimize the factor of order of presenta- 
tion within each session and from one 
session to another. 

Stimulating conditions. The lights 
whose effects upon electrical sensitivity 
were measured in this experiment were 
presented by means of the optical system 
previously described. The 2*8' field was 
used, and the duration of the light was 
two seconds. The intensities of the red, 
green, blue, and white light were equated 
at approximately 13 ft-L. by the use of 
neutral density filters. The specific red, 
green, and blue colors were achieved 
by the use of the selective filters de- 
scribed in the apparatus section of this 
paper. The delay between the end of the 
two-second light flash and the electrical 
stimulation was set at 1, 2, OT 3 sec. 
ixperimental session. During a single 
session in this experiment, the threshold 
following stimulation with one color was 
determined for each of the three delay 
times. The procedure used for a single 
session is described in the preceding 
section. The LC method was used to de- 
termine all thresholds. The initial elec- 
trical stimulation in each threshold de- 
termination was kept at the same general 
level for the light and dark determina- 
tions. Each descending series of electrical 
stimuli following flashes of light was 
started at the same intensity as the pre- 
ceding dark. The starting intensities for 
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the dark determinations were decreased 
or increased by one step in a systematic 
fashion through the four determinations 
of a session. 'The 5s were unaware of this 
manipulation of the intensity with which 
the descending series began. 
Experimental design. Four Ss partici- 
pated in Experiment II, which consisted 
of eight sessions. Each of the three colors 
and white light were used in two ses- 
sions. The delay times were presented in 
the order 1, 2, 3 sec. for one of these ses- 
sions, and 3, 2, 1 sec. for the other ses- 
sion, for a given color. The colors were 
presented in an ABCDDCBA order 
through the eight sessions. Each of the 
four Ss began the series with a different 
color. The first delay time used in the 
session, 1 or 3 seconds, was alternated for 
the eight sessions. That is, in Session 1 
the first delay time was 1 sec.; in Session 
2, it was 3 sec.; in Session 3, 1 sec., etc. 
Two of the Ss started with a 1-sec. delay 
in Session 1 and two began with a 3-sec. 
delay. 


Results 


Effects of delay time on enhancement. 
The results of Experiment II are ex- 
pressed as enhancement values (є) com- 
puted from each of the thresholds follow- 
ing stimulation by light. The data of 
each session are presented separately. Fig- 
ure 6 presents the two replications for 
each color for each of the four Ss. The 
abscissa of each of the 16 graphs pre- 
sented in Fig. 6 is the delay time, i.e., the 
time in seconds between the end of the 
2-sec. flash of light and the onset of the 
o.1-sec. pulse of electrical stimulation. 
The ordinate of each is ¢, the enhance- 
ment of the electrical sensitivity by light 
as defined above. The points designated 
by xs represent the enhancement values 
obtained during the first session with a 
given color and the os the second session. 


Each enhancement value is based on а 
single threshold in the "light" condition 
as compared with two “dark” thresholds, 
ie, those immediately preceding and 
following the "light" series. 

It should be pointed out again 
Motokawa reports that the maximum en: 
hancement effect for blue light reliably 
occurs at 9 sec., for green and white light 
at 2 sec., and for red light at 1 sec. The 
data in Fig. 6 show no such consistent 
"crest times." It is also evident that the 
light does not consistently have the Шу; 
of enhancing the sensitivity to electrica" 
stimulation. Instead, many of the points 
in Fig. 6 show “negative enhancement 
and many are approximately zero. Ше! 
LAR is the only one who shows СО!” 
sistently “positive enhancement.” 

In general, for all four Ss, the enhance 
ment resulting from stimulation by blue 
light is greater than that from the 
two colors or white light, whereas reos 
green, and white have approximately 
equal effects within the individual sub- 
jects. а аб 

Variability. It is evident (тот Fig. Е 
that there is а good deal of session-toer 
sion variability in the enhancement сна 
ues for a given color and delay time: j " 
each color and delay time, the tw? b 
hancement values obtained on sepat e 
days were indicated by xs and 05. Ta om 
8 gives the mean discrepancy for all © se 
responding values of e. Each of ve 
means is for differences, both арі 
and algebraic, together with the stan. Also 
deviation of the algebraic values. : e. 
given is the mean of the discrepancies 
tween “dark” thresholds obtained Ре т 
and after each light threshold. It p n 
seen from Table 8 that there is nO s 
sistent tendency for the enhance" 
value obtained on the second day differ 
higher (i.e, with a positive mea? “ gjf- 
ence) or lower (with a negative mean 
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> 
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Red 
1 2 3 

~ 
u 

Green 
= $— $3 
Ф 1 2 3 
Е 
Ф 
о 
(= 
О Blue 
<= 
5 
ш 

White 


Delay Time (sec) 


Fic, 6. Enhancement effects resulting from flashes of red, green, blue, and white lights of equal 
ted by xs are for the first session 


photometric luminance in Experiment п. The points designa 
1 session. The "delay times" of 1, 2, and 3 sec. are measured from the 


and the os for the second 

end of the 2 sec. flash of light to the beginning of the 0.1 sec. pulse of direct current, The light 
is presented as a 2 og' field at approximately 1g ft-L. Note that there is little agreement between 
these results and those of Motokawa as shown in Fig. 1- Where enhancement occurs at all it 
fails to conform to the specific wave- -length curves that Motokawa has presented. 


ference) than that of the first day. dark thresholds obtained before and after 
ach light are also seen in Table 8. In 


The average discrepancy in the two 


TABLE 8 


or THE DATA OBTAINED IN EXPERIMENT II 


VARIABILITY O 
(T К " d in this table are based on all 4 colors and 3 delay 
he differences кошш er ae going into each mean for each subject) эченд, 
making а ota 
2 Е Я Mean Discrepancy 
" U der ean 
ЕК | Dust Threshold 


ifferent Days 
Surrounding Dark Discrepancies of 


Subject the Same Petitions on D 
Thresholds in Units 
SD of^2009teps s Steps or Less 


Mean Difference Mean Difference 
lc (Absolute) WW. me ў 53% 
Е c^ : p 1-1 o 
SHH eap ne ‚одо ee HA 
EE ш е п 
18 -950 —9 ww c5 | Ae 
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the section of this report on evaluation 
of the LC method, similar measures of 
discrepancy were presented for Ss LAR 
and SHH. In that study, however, the 
thresholds compared were immediately 
adjacent, ie, without an intervening 
light threshold. Under those conditions 
lower discrepancies were found, i.e., 4.77 
steps in the earlier study as opposed to 
7-71 steps just reported for LAR, and 341 
as compared to 5.17 for SHH. The in- 
creased discrepancies probably reflect the 
effects of the longer time interval be- 
tween dark determinations. For $s PMG 
and JS, who did not take part in the ear- 
lier study, the mean discrepancy in the 
present study is less than five steps. 

An important consideration in com- 
paring these data to those of Motokawa 
is in the number of pairs of dark thresh- 
olds, preceding and following light, in 
which the discrepancy is less than 10 per 
cent, or approximately five steps. These 
figures also appear in Table 8. Motokawa 
discards data where the discrepancy be- 
tween dark thresholds is greater than 10 
per cent. It is evident that Motokawa 
would have rejected most of the data of S 
LAR, who showed the greatest enhance- 
ment values, but had only 8 out of 24 
pairs of dark thresholds differing by five 
Steps or less. On the other hand, 21 out 
of 24 of the enhancement points of PMG 
аге based upon dark thresholds 
Vary by five steps or less, 
not show consistently pos 
ment for any condition. 

Conclusions with regard to enhance- 
ment as affected. by delay time with а 
small central field. We may sumarize 
the results of Experiment 1I, then, by 
stating that (а) positive values of en. 
hancement were not universally obtained 
in this experiment, (b) there was a 
marked tendency for higher enhance- 
ment values to occur with flashes of blue 


which 
yet this S did 
itive enhance- 
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light than with white, green, or red indie; 
all flashes were of approximately equa 
photopic effectiveness, (c) only апе а> 
ject (LAR) showed consistently positiv 

enhancement values, and he was the 
most variable in his performance of all 
four of the Ss, and (d) there is no consist- 
ent tendency for red light to yield ре 
greatest enhancement at a delay time я 
1 sec., green light at 2 sec, or blue lig 
at 3 sec. 


Experiment HI. Enhancement as A acf 

М К o0 
tion of Intensity of the Preceding Flash Ө) 
Light from A Small Central Field 


The preponderance of vou 11 
values of enhancement in Experiment li- 
suggests that some of the specific pe 
tions of the experiment may not не 
been optimal for enhancement ee ur 
occur. ‘This is in spite of the fact that s “A 
conditions appeared to аргон, 
those used in the studies by Маон 
Accordingly, it was decided to vary vem 
aspects of the stimulus in order 50 m 
the possibility that other audi 
might be more favorable. Motokawa oe 
reported that the degree of ene 
is positively related to the intensity ei тс 
preceding flash of light. It was there pe 
decided to use the three Ss who A i 
relatively little enhancement in EXP ce 
ment IT, and find the degree oren 
ment when intensity was varied. 1 
light and blue light were empio oi 
Experiment III and three different 1n de- 
sities of each were presented. A MR 
lay time was used for the red and 2 3 e- 
time for the Blue to favor a maximum ^ 
gree of enhancement in accordance V! 
the reports of Motokawa. 


small 


Procedure 


li- 
- ‘ "T T cont 
Stimulating conditions. The 


Ы E А за 

tions of Experiment III were ane ез 0 
ө à А ash 

those of Experiment 11. The flas 
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PMG 


LOG I 


Vic; 7. Enhancement in relation to l 


5°8' field, used in Experiment IIL. 


light were seen as a 278^ field presented 


to the dark-adapted right eye of the S. 
Red (С) and blue (76) filters were used, 
and the variation in intensity of each 
1.0, ОГ 2.0 


was achieved by inserting 0.0, 
into the 


log units of neutral density filter 
stimulating system. 

During any one session, a single color 
Was presented at each of the three inten- 
sities, 

Experimental design. Vhree Ss served 
nt for four sessions each. 
‘There were two sessions ОП each of the 
two colors, red and blue. During one of 
the intensities 
1.0, and 


in this experime 


the sessions on each color, 
Were presented in the order 0.0; 
2.0, and during the other this order was 
reversed. The three Ss were those who 
showed least consistent enhancements in 
Experiment II. 
Results, Figure 7 pre 
Experiment III for each session for each 
S. The points marked with xs represent 


the first session on a given color, and 05 


sents the data of 


ight intensity. 


^2 
o 
o 
te к. ЖЕ Чыр 
= 
-£ -1 0 -2 -l 0 
LOG I LOG I 


Data for three different intensities of light, 


the second session. It can be seen that 5 
SHH shows an increased enhancement 
value with increasing intensity of light 
on all four sessions, with both blue and 
red light. The S JS has a clear intensity 
effect with blue light and on one session 
with red light. The S PMG, on the other 
hand, appears to have larger enhance- 
ment values for the lower intensity of red 
light, and the blue light has an irregular 
ellect. Again in this study, the variability 
of the enhancement values under similar 
light conditions is high for all Ss. The 
variability of the dark thresholds preced- 
ing and following the light thresholds is 
of the same order as in Experiment II. 


Experiment IV. Enhancement Produced 
Ву Matched Red and Blue Fields 


Experiments 11 and III have yielded 
the information that blue light often 
produces considerably greater enhance- 
ment than does red of comparable photo- 
metric intensity. This finding suggests 


20. 2 


that, despite the use of a small central 
field of stimulation, the effectiveness of 
these lights may be related to the extent 
to which they affect the rod, or scotopic 
response mechanisms of the retina. Ex- 
periment IV was set up to give further 
information about this point, and to pro- 
vide further comparisons between delay 
times of 1 sec. and 3 sec. 


Procedure 


Stimulating conditions. In Experiment 
IV, two intensities of blue light and one 
of red light were used in each experi- 
mental session. The brighter blue light 
is a photopic match to the red, each flash 
having a luminance of about 13 ft-L. 
The other blue light is less intense by 
about 2 log units; but its scotopic effec- 
tiveness is equivalent to that of the red 
light. 

The light was presented as a 2?8' field 
for two sec. The delay between the end 
of the light and the onset of electrical 
stimulation was either 1 sec. or 9 sec.; it 
was maintained at one or the other delay 
time throughout any given session. 

Experimental design. Three Ss partici- 
pated in four sessions each for this ex- 
periment. In each of the four sessions, 
the two intensities of blue light and the 
red light were presented. For two of the 
sessions, the delay time was 1 sec. and for 
the other two it was 9 sec. A single ses- 
sion began with one intensity of blue 
light, followed by red light, and ended 
with the other intensity of blue. The in- 
tensity of the initial blue light was varied 
in an ABBA order over the four sessions, 
while the delay times, 1 sec. and 3 sec., 
were alternated. 

Results. Figure 8 presents the enhance- 
ment values obtained in each session. As 
before, the first session is indicated b 
xs and the second by os. The results for 
the 1- and 3-sec, delay times are pre- 
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sented separately. It is apparent that the 
less intense blue yields enhancement val- 
ues more like those of the red light than 
does the more intense blue. In general, 
enhancements are greater for the more 
intense blue than for cither the red or 
the dimmer blue. This suggests that the 
effectiveness of these lights may be more 
closely related to their scotopic effec- 
tiveness than to their luminance 1n foot- 
lamberts as given by a photopic fieron 
For SHH and JS there is a clear increase 
in enhancement as intensity meres: 
supporting the earlier data on this pem 
For PMG, however, intensity does no 
have any consistent effect. Pa 

A comparison of the enhancements 3 
sulting from the 1 sec. and 3 sec. delay 


: РР У 

times does not yield the findings po 
А мыт а 

would be predicted by Motokawa. б 


is, there is по consistent tendency ip 
enhancement to be relatively greater ү 
з sec. for blue light and at 1 sec for Bn 
The data appear to be slightly less d 
able with a i-sec. delay than with a 3% 
one. 

The variability of the dark threshold 
in this study is similar to that ама 7 
earlier. For two Ss the average —-^ 
ancy between dark thresholds mm. i 
and following the light thresho 
slightly less than five steps, OF 10 At 
cent, whereas this discrepancy i$ app” ; 
mately seven steps for the third 5, . 


ША 
1 гоо! 
Experiment vs. Enhancement Fo 


reld ~ 
Flashes of Light from a Large Field 


; 11, a" 
The results of Experiments IL Í gua y 


IV have shown that blue light е elec 
produces a greater enhancement pe гей 
trical sensitivity than does gree? ınan 
light of equal photometric Ium nat the 
This has led to the supposition ^. may 
enhancement due to a flash of PE alat 
be governed by its ability tO с тів! 
the rod or scotopic receptor syst? 


| 
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Fic, 8. Enhancement produced by m 
mately the same luminance (13 ft.-L.) 
15 approximately that of the dimmer blu 
and delay times of 1 sec. and g sec. 


this should be true for stimulation by a 
small, central field of light may at first 
appear to be unlikely. Work with the 
human electroretinogram has clearly 
shown, however, that scotopic effects may 
arise in this way. Indeed it is found that 
it is difficult, if not impossible, to obtain 
any electrical responses from the stimula- 
tion by light of small central areas (Ad- 
rian, 2, g; Riggs and Johnson, 24)- Such 
responses as do occur in the human eye 
under these conditions have been shown 
by Boynton (4) to originate primarily 
from the action of stray light on pe- 
ripheral scotopic receptors. These facts 
suggest that the peripheral regions of the 
retina occupy a preferred location, elec- 
trically, so that diffuse electrical stimula- 
tion of the eye by large electrodes has 
primarily peripheral effects. 

If the enhancement resulting from 
small-field stimulation is due primarily 
to the action of stray light on peripheral 


PMG JS 


+2 
х 
5 о +) 
ڪ و ڪڪ 0 ي‎ 
x x о х 
-l o 
Res Be o Res Be % 


atched тей and blue fields. The red light, Кз, has approxi- 


as the brighter blue light, Bo 
e light, B... Results of Experiment 1V with a 2?8' field 


The scotopic ellect of the red 


receptors, then it should be true that an 
even more effective means of securing 
enhancement would be to extend the size 
of field so as to provide a more direct 
stimulation of the rod receptors. Further- 
more, most Ss report that the phosphenes 


aroused by weak electrical stimulation 


are not localized in the central so much 


as in the peripheral field of view. It 
seems appropriate, therefore, to use 
large-field stimulation in order to affect 
by light those regions of the retina that 
appear to be affected by electrical stimu- 


lation.® 


—À 


s We have found that when the intensity of 
the large field is reduced to near-threshold 
levels it becomes impossible for the $ to distin- 
guish between the “phosphenes” produced by 
photic and electrical stimuli, provided that the 
photic stimulation is in the form of a large field. 
Either form of stimulus results in a vague, color- 


Jess cloud of light. 
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Procedure 


Stimulating conditions. For Experi- 
ment V, the light was presented as a 38° 
field, by means of the special eyepiece 
described in the apparatus section. Red, 
green, and blue lights, equated for sco- 
topic effect, were used with central fixa- 
Hoa 

‘The delay time between the end of the 
light flash and the electrical stimulation 
was varied from .2 sec. to 4 sec. in order 
to cover the range of delay times re- 
ported by Motokawa (15) and by Mita, 
Hironaka, and Koike (13). 

Experimental session. A single session 
in this experiment involved the presenta- 
tion of a single color at four or five of 
the delay times. The duration of a single 
session was limited to 114 to 134 hours, 
and this factor determined the number 
of enhancement values that could be 
obtained. The delay times were divided 
into three categories. Category I included 
the short times (0.2, 0.4, 0.6, 0.8, and 1.0 
sec.); category II included medium times 
(0.8, 1.0, 1.25, 1.50, and 2.0 sec.); and 
category III included times in integral 
seconds (1, 2, 3, and 4 sec). On a given 
session the times from one of these cate- 
gories were presented. Since the cate- 
gories overlap, we have obtained several 
determinations for some of the delav 
times but only single determinations foi 
others. 

Experimental design. Four Ss partici- 
pated in this experiment. Two of them, 
LAR and SHH, had already observed in 
the small field experiments. The other 
two had had no experience with the 
small field, but were given prelimin 


ary 
training in the observations 


of phos- 

” Equal scotopic effectiveness was achieved for 
the C, G, and 76 filters, designated in this 
by R, G, and В, by the use of neutral density 


filters, As used here, the scotopic effectiveness 
is approximately equal to that of white at 3 ft-1 


paper 
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phenes with the 38^ light field. В 

Each 5 required at least nine Seo 
to complete the exper iment, since there 
were three colors and three ume cate 
gories and each session involved а sep 
rate one of the nine possible combini 
tions of time and color. 

The order of presentation of the de 
lay times within a single session was such 
that the shortest and longest delay Нони 
(within the category being presented) 0€ 
curred near the middle of the session: 
Thus the session began and ended with 
intermediate rather than extreme values 
of delay time. | " 

In general, all times categories for a s 
gle color were presented in consecutive 
experimental sessions. For example, p 
S started with three sessions in which I 
light was used, followed by three of eres’ 
and three of blue. ‘The order of present 
uded 
The 


tion of colors and categorics is incl 
in Table 9 in the results section. [a 
time category for the first session 9 
given color was randomized, as was 
order of colors presented. 


Results 


Effects of delay time on enhance’ E 
Table 9 shows the principal ME i 
Experiment V. It is seen that the st 
hancement values are negative for mes 
yield predominantly positive enb thet 
ment for two of the Ss, but for the © 
two the enhancement values are ae 
tially zero. In Fig. g, an attempt is d 
show these findings graphically. 100 
number of experimental points ba 
great to be shown individually ue 
figure. Accordingly, all the € valu 
ferring to the same condition 0 Jil d 
time, $, and color have been — 
by averaging to a single value T 
graph? i 


ox 


this em 


this 


Our major conclusion from 
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TABLE o 


ENHANCEMENT VALUES (€) FOLLOWING FLASHES or LIGHT FROM THE LARGE FIELD 
USED IN EXPERIMENT V 


Sub- Ses. Delay Time (sec.) 
oio Color | = 
ject ston ч 
2 E о 5 1.0 1.25 1.50 2.0 3.0 4.0 
Green | 4 | —.251 +.017 F.076 1-140 
$ Tai; +153 4.077 1-105 
6 +.126 +.165 +.005 +.126 
SMM) Red | 3 | —.326 Hin Hass tau 
1 —.00; 4.218 4.178 4.170 
2 +.229 4.145 4.100 4.118 
Blue 8 | —.270 4.177 +.205 +313 
о +.256 +.222 +.2901 4.271 
7 | 4.270 +.201 4.208 4.250 
Green | 3 |—.149 —.038 4.012 +-000 +.017 
2 —.010 +.032 +.017 —.036 +.028 
1 +.031 +.008 —.012 +.025 
MC | Red g | —.342 —:075 —:057 —:080 — o18 
0 —.058 —.009 —.002 —.046 —.044 
4 —.014 —.013 —.030 +.010 
Blue 7 | —.375 —121 —.246 —.000 —.o84 
8 +.019 +.083 —.043 —.011 —.030 
6 +.030 осо —.032 —.040 
Green | 5 | =.364 —.210 —.175 —.120 —.114 
6 —.078 ‚соо —.070 —.021 +.017 
7 —.o86 —.021 —.020 —.077 —.116 
CJW | Red io =a Rg age es TTR 
9° —.0;8 —.056 —.005 -057 = o81 
8 —.074 +.003 +.128 
2 —.032 т +.017 +.023 
Blue 3 | —.2t1 —.170 —-078 —.046 —.057 
—.004 —.045 —-022 —.017 —.043 
i +.048 +.032 +.042 
Green | то | —.061 .00 
reer 10 э 4.000 4.772 4.225 
3 4-.ото +.185 +.024 
ы фе = —.0 
AR | Red 8 Jut 49 gara А403 
B +.084 +111 +.057 
„ех 4.269 4:242 41-304 +.206 
Blue ; aay “+1489 F341 4.247 +104 124 +.225 
2 .006 +.184 -b.124 
6 T ue em 4.236 +188 +.262 
1 -L.28o +.209 +.242 +.255 
4 


"The averaging has been done for conveni- 
ence in presenting the results, and it must be 
Tealized that there is some variability in the 
Points going into these averages. The individual 
Points are found in Table 9- These points sup- 
е equally well our conclusions with regard 
to enhancement as a function of delay time: 


eriment is that beyond about one sec 
ond of delay time there is little relation- 
ship between delay time and degree of 
enhancement. This conclusion applies 
to each of the colors used; i.e. there is 


no apparent ‘crest time" as а function of 


а s көл. Ж бё. 
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a ; of 
Fic. 9. Enhancement as a function of delay time for blue (e), green (о), and red 9) eus Se 
equal scotopic effectiveness in Experiment V. The light is present as a 38° ficld whos Figure 
effectiveness is similar to that of white light at g ft.-L. Compare with Motokawa's results od e 
1. Note that enhancement is clearly present in but two out of the four subjects; and pro eis 
fail to conform to the pattern predicted by Motokawa. АП four subjects, however, show clea 
dence of reduced electrical sensitivity immediately following the light flash. 


the wave length of the light. There are 
no marked differences in effect among 
the three colors used here. As before, 
however, the blue light does appear to 
have a greater effect than the red or 
green, especially in the cases of the two 
5s, LAR. and SHH, who showed large 
positive enhancements. This is in spite 
of the fact that the blue light is much 
less bright, in terms of photopic lumi- 
nance, than the other colors. Since the 
three colors are approximately equal in 
their scotopic effectiveness, it appears to 
mean that the degree of enhancement is 
approximately what one might predict 
from a rod-receptor mechanism, but that 


blue is slightly more effective than would 
be expected.'? 


* This finding may perhaps be the result of 
Rayleigh scattering which, in the case of the 
human electroretinogram (Boynton, 4) has often 
given the appearance of excess blue sensi- 
tivity. Blue light is very effectively scattered by 
the optic media, so that it exerts a dispropor- 
tionate influence on functions involving the 
extreme periphery of the eye. 


Data on threshold variability. AS he 
the previous studies, we have nee 
the number of steps separating the ee 
reference thresholds used to compute = 5 
enhancement effect of each light. pss 
number of such pairs for each 8 PED 
periment V is from 96 to 51. Fora ave 
the average discrepancy is less than wa 
steps, or approximately 10 рег A me 
Table 10 presents these averages in 
gether with the percentages of par or 
which the discrepancy was five steps 
less. 


TABLE ro ERENCÉ 
r5 БЕЕК" 
AVERAGE DISCREPANCY BETWEEN a 
‘THRESHOLDS PR DING AND $ 
LOWING A LIGHT THRESHOL 


Gn 2 per cent step units) 


= = © 
LAR знн СМ سے‎ 


Average Disctep- 15 


T 

ancy 3.33 ad p oem 
40 

N 51 36 40 


Percentage of 
Discrepancies of 79 64 ee 
5 Steps or Less. 
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The variability in dark thresholds in 
this experiment was in general smaller 
than had been found in the earlier 
studies. The reasons for this are not 
clear. Two of the Ss were highly experi- 
enced and two were relatively inexperi- 
enced. It is of interest that most of the 
pairs of reference thresholds (from 64 
per cent to 79 per cent of them in the 
various Ss) had small enough discrepan- 
cies so that they would have been ac- 
cepted as valid by Motokawa's criterion 
of reproducibility within 10 per cent. 


Experiment VI. Enhancement as a Func- 
tion of Intensity of Light from a Large 
Field 

Experiments И, Ш, and IV demon- 
strated that with some Ss the degree of 
enhancement is positively related to in- 
tensity of light, as Motokawa has re- 
ported with a small, central field of light. 
Experiment V has demonstrated high 
values of enhancement for two of the 
four Ss when large-field stimulation is 
employed. Experiment VI is one in which 
a large field of white light is presented 
at three different levels of intensity. The 
aim of this experiment was to maximize 
the enhancement effect by utilizing. a 
large field of light, and to study the in- 
fluence of delay time and intensity on 
enhancement under these conditions. 


Procedure 
АП the data 


Stimulating conditions. 
ere obtained 


included in this section W 
with the large (38°) field of light pie 
ceding the electrical pulse. Three inten- 
sities of white light were used. The 
middle intensity was the scotopic equiv” 
alent (with 2.6 log units of neutral den- 
Sity filter) of the colors presented in Ex- 
periment V, Its photometric Juminance 
Was approximately 3 ft-L- The other two 


intensities were two log units above and 


+4 


+ 
ә 


° 


Enhancement (€) 


Delay Time (Sec) 


Fic. 10. Enhancement as a function of delay 
time for white lights of approximately 300, 3, 
and 0.03 ft.-L. luminance. Subject LAR, Experi- 
ment VI. Two other subjects failed to show 
consistently positive enhancement under the 
same conditions. 


below that value. Each flash of light was 
followed, as before, by an electrical test 
stimulus at a delay time ranging from 
0.2 sec. to 4 Sec. 

Experimental design. The design of 
Experiment VI was similar to that of 
Experiment V. In a given session, only 
one intensity of white light was used, at 
various delay times. Three Ss were used. 

Results. As in previous experiments the 
thresholds for electrical stimulation were 
found to be high immediately following 
each flash of light. In other words, the 
value of s was negative for delay times 
below about one sec. With longer delay 
times one $ (LAR) showed positive values 
of є as in earlier experiments. Two other 
Ss failed to show any positive enhance- 
ment effects. 

Figure 10 presents the time course of 
enhancement for three intensities of 
white light for the one S whose electrical 
thresholds are markedly affected by the 
lights of various intensities. As in the 
previous experiment, some of the points 
on these curves represent the averages of 
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two or three determinations, while others 
are single values. It is clear that, for this 
S, the degree of enhancement 15 а func- 
tion of the intensity of the light preced- 
ing the pulse. The most intense light 
vields enhancement values consistently 
oreater than those for the other two in- 
tensities at all delay times except 0.2 sec., 
where the depressing effect of the intense 
light is greater than for the other two. 
The absolute values of these enhance- 
ments for the most intense light are 
higher than any previously obtained, and 
appear to reach a maximum in the neigh- 
borhood of one sec. after the flash. Some 
tendency toward the one-sec. maximum is 
also shown at the middle intensity of 
white light in this figure and in the most 
intense blue light of Fig. 0. 


Experiment ҮП. Enhancement Produced 


by Large Fields of Colored Lights of 
Equal Luminance 


Intensity of white light has been 
shown to determine the degree of en- 
hancement in a $ who readily shows en- 
hancement effects. For colored lights it 
is clear from tne data of Experiments II, 
III, IV, and V that this intensity should 
be evaluated not in photometric units of 
luminance but rather in terms of the 
effectiveness of the light for stimulating 
the scotopic system of the eye. Experi- 
ment VII is one in which this conclusion 
is further verified by the use of colored 
lights of approximately equal photopic 
luminance. 

Experiment VII consisted of present- 
ing blue, red, and green lights whose in- 
tensities were equivalent, photopically, 
to a white light of about 0.2 ft-L (with 
3.6 log units of neutral density filter). 


These lights were followed by electrical 


stimulation at i-, 2-, 3-, and 4-sec. delay 
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` E d 1 o 
Fic. ıı. Enhancement as a function ni of 


time for blue, green, red, and white lig LAR, 
approximately 0.2 ft..L. luminance. srt ing hate 
Experiment VI. A second subject em 
greater enhancement for blue than for t Ше ts in 
colors, but failed to show consistent effects 
relation to the variable of delay time. 


А , ey periments 
times. From the results of Exper 


II, III, IV, and V it would be predicted’ 
that these lights, though of equal ee 
ance, would yield different amounts i o 
hancement. Thus blue light, Mm, 
its relatively greater effectiveness а, 
arousing the peripheral scotopic pase 
would be expected to produce BY 
enhancement than green, and 
greater than red. [бї 
Figure 11 presents the results (en 
one 5 (LAR) who showed consisten plue 
hancement effects. As predicted, ed 
light is most effective in enhancing 
electrical threshold, green light 15 are 
effective, and white and red ЦЕП es 
least effective. The points on these x he 
represent single determinations, a 15€ 
three curves were obtained оп о! 
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а h isht W 
tive days. The curve for white r Jate” 
; с о wee 3 
obtained approximately two we aria’ 


. ке more V 
The data for another 5 were nt the Jue 
ble but confirmed the fact that 15 
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EFFECTS OF ELECTRICAL STIMULATION AT 
SUBTHRESHOLD INTENSITY LEVELS 


The Concept of Multiple Thresholds 


A central concept in the thinking and 
experimentation of Motokawa is that of 
multiple thresholds. This concept is to 
the effect that several electrical thresholds 
corresponding to the several fundamental 
response mechanisms of the retina may 
manifest themselves at once in the re- 
sponses of the S. No provision for such 
a phenomenon is made in the usual psy- 
chophysical experiments. In fact, there 
appear to be certain logical difficulties 
implicit in the concept. Nevertheless, the 
stress laid on multiple thresholds by Mo- 
tokawa and the revolutionary nature of 
this idea make it important that they be 
given serious consideration in relation to 
our procedure and results. 

Specifically, Motokawa assumes that 
several electrical thresholds may exist at 
any one time, one threshold for each of 
the fundamental response mechanisms 
affected by the preceding flash of light. 
Of these multiple thresholds the true one 
is assumed to be the lowest, i.e., the one 
below which no phosphene is ever re- 
ported by the S. Motokawa (20) has in- 
sisted that great care is necessary tO at- 
tain the lowermost or true threshold. In 
his words, “Trained Os can easily pass 
apparent thresholds and reach the true 
threshold, but untrained Os stop at one 
of the apparent thresholds, because they 
cannot discriminate sufficiently slight 
Phosphenes from the background." Ап 
inexperienced experimenter may fail to 
continue presenting the stimuli in the 
descending series until the lowermost 
threshold is attained. 


i e 
In order to avoid the danger of Ud i 
determination at one of the apparent thresholc 


and not attaining the true one, Motokawa (21) 
has suggested the following procedure. When the 
threshold criterion has been reached, he presents 
further comparison pairs at an intensity level 
about 5 per cent lower than the threshold level. 
If the S can discriminate at this lower level, 
this is taken as an indication that the threshold 
attained was not the true one, but an apparent 
one, and the determination is continued until 
no lower discrimination is possible. More re- 
cently, Motokawa has devised another means of 
avoiding apparent thresholds. In a 1955 paper 
(22) he makes the following statement of his 
procedure: "In our routine work, we try to 
determine a true threshold alone, skipping over 
apparent ones by graduation of voltages in gross 
steps, but not forgetting that the obtained 
threshold is really a true onc." 


There appear to be certain logical diffi- 
culties in the concept of multiple thresh- 
olds. Let us assume that the blue re- 
sponse mechanism, for example, accounts 
for the lowermost or “true” electrical 
threshold and the red response mecha- 
nism has a somewhat higher electrical 
threshold under certain experimental 
conditions. Does this not mean that an 
electrical stimulus whose intensity is 
equal to that of the red response thresh- 
old will stimulate even more strongly 
the blue response mechanism? In other 
words, is it not difficult to conceive of a 
psychophysical relationship showing any- 
thing but a decreasing frequency of “уез” 
responses as stimulus intensity goes down, 
even though several different response 
mechanisms may be affected simultane- 
ously? 

In our own experiments the effects of 
light have been specified in terms of the 
change that the light produces in the 
threshold for electrical stimulation of the 
eye. Each threshold has been determined 
by the LC method as specified above in 
the section on general procedure. It 
must be emphasized that this method has 
been rigidly imposed in all cases, and 


that equal log steps have always been 
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A. Total Number of Steps Between Original and Final Threshold 


Dark Condition 


Light Condition 


Total 
I 2 3 4 l'otal a 2 НЕ a i 
r р 8 Y ( 5 25 
LAR 6 19 15 18 58 9 5 > A 25 
SHH ти 17 B 38 үс 1 7 0n o 3 
PMG 9 10 14 9 42 9 d 5 хо 40 
CJW 10 9 20 9 57 13 D bos 
a 122 
Total 205 = — 
Cm 5 < 3s 3 sri ا‎ Threshold 
B. Number of Steps Correctly Judged Between Original and Final Ihc hok 
~ Dark Condition Light Condition 
ч күч М А Total 
1 3 4 Total I 2 3 4 m 
mx CERA um | 5 13 
LAR 3 1$ 6 7 30 4 3 + í 
SHH 4 8 13 7 32 5 1 2 à 13 
РМС 6 4 8 4 22 I o 3 à 20 
CJW 6 4 19 5 34 9 7 5 Б 
РИ 02 
Total 124 —= 


extended threshold determinations was 
begun. The first and third of these were 
dark, or reference thresholds, and the 
second and fourth were light thresholds. 

Results. The results of this experiment 
are of interest chiefly as comparisons of 
the dark and light conditions. Two such 
comparisons are presented. Both are 
based upon the responses made to stimuli 
below the level of the original threshold. 
That is, interest here is centered upon 
what happens during the special exten- 
sion of the comparison stage of the LC 
method. 

First, let us consider the number of 
two per cent steps between the original 
threshold and the termination of the ex- 
tended. threshold determination. It will 
be noted that the minimum number of 
steps between these two criteria is five, 
since the extended series w 
inated until at least five steps below the 
first threshold had been presented. Sec- 
ond, we may note the number 
correctly judged between the or 
final thresholds. No minimum 


as never term- 


of steps 
iginal and 
number of 


S 
" sura. codice: THE 
steps exists for this measure, sinc 


Fuss hi8 
may make no correct judgments po 
original threshold has been reachec ' with 
measure, then, reflects the Success : at 
which the $ was able to discriminzie as 
intensity levels below that which yeri- 
called threshold in the earlier ai? ide 
ments. The data in Table 11 118 ^ 
both of these measures. Because prie 
ditions (field size, delay time) E 
slightly among the four $5, in eig 
data are presented in Table 11. ^ were 
thresholds for any given 5 hoe лө 
obtained under identical cond! show 
Both measures given in Table es ina 
that Ss are often able to pret 
phosphenes from blanks at 1 Id i 
levels below the original thresho: 
both the light and dark вш 
in no case does the light condition © guC 
the dark condition in the number 0 dar* 
cessful discriminations. Instead, the nin“ 
condition seems to favor such pape 
tions, yielding about twice as many poth 
when all Ss are combined, Í?! he 
measures, This result is in contrast ‘ 


put 


" 


prediction of Motokawa (22) for the mul- 
tiple threshold. effect. His prediction is 
that of а more abrupt transition from 
positive to negative responses in "dark" 
determinations as opposed to “light” de- 
terminations. 


Experiment IX. Employment of the CS 
Method with Small Steps of Intensity 
Above and Below the Threshold 

. The results of Experiment VIII have 
indicated that phosphenes аге sometimes 
aroused by stimuli below the usual 
threshold, but that this is not brought 
about in any unique fashion by the after- 
effects of flashes of light. The conven- 
tional concept of a threshold demands, in 
fact, that some positive responses be ob- 
served when stimuli of subthreshold in- 
tensity are given. Accordingly, Experi- 
ment IX was conducted to quantily the 
relationship between the frequency of 
positive responses and the intensity of 
stimulation over a wide range of sub- 
threshold and suprathreshold intensities. 

Procedure, The method of constant 
Stimuli was used in Experiment 1X. Nine- 
teen intensitics of electric current were 
presented 20 times each. These intensities 
differed by steps of two per cent, as was 
true of the steps used in the LC method. 
А blank stimulus was also presented 20 
times, making a total of 400 presenta- 
tions. A rest period of about one minute 
followed each 4o present 
dom order of 300 stimuli, 
Intensity 10 times, was presented twic 
Opposite directions. The 5 respondec 
cach stimulus by saying “yes” OF "но 
and the experimenter made no comments 
€xcept to identify the blank stimuli after 
they had occurred. 

A ssec, blue light (7 
Units N. D.) preceded each 
Stmulation by a delay time of 
In all of the experiments in W 


ations. А ran- 
including each 
ein 
1 to 


6 plus 10 log 
electrical 
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hich light 
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Fic. 12. Percentage of "yes" judgments for 


rance of phosphenes in relation to 


the арр 
intensity of the stimulating current. 

Method of constant stimuli with 2 per cent 
steps. Subject LAR, Experiment IX. This experi- 
ment was attempted with two additional sub- 
jects. Their data failed to satisfy the usual 
criterion of 75% or more judgments of “seen” 
at the high end and 25% or fewer at the low 
end. Presumably this is а consequence of the 
length of the experiment (215 hours) and the 
small size of step (2%), factors that render the 
experiment an unusually arduous one for the 


subject. 


was presented, 15.3 sec. separated succes- 
sive stimulations. The small (2° 8’) field 
of light was used, with central fixation. 

The session was begun with a prelim- 
inary threshold determination by the LC 
method without light, followed by an- 
other preliminary LC threshold with the 
same intensity and delay time as was 
used in the CS determination. The CS 
determination was begun at approxi- 
mately 30 minutes after the S entered the 
dark room, and it took approximately 
two hours to complete the series of stim- 
ulus presentations. 

Results. Figure 12 presents the con- 
stant stimulus function. for S LAR, 
on probability paper. The ab- 


plotted 
log intensity of current and the 


scissa 18 
ordinate is per cent scen plotted on a 
scale of probability. The blank stimulus 
was never reported as seen during the 
resentations. The straight line has 


20 p 
fitted to the points by the method 


been 
of least squares. 
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Despite the small range of intensities 
used here (0.157 log unit), a wide range 
of response frequencies was obtained. 
Each of the three lowest steps yielded 25 
per cent or fewer judgments of “yes” 
and the seven highest intensities were all 
“seen” on 75 per cent or more of the 
presentations.? Although there are ге- 
versals in the function, there is по evi- 
dence of major dips and peaks, as would 
be predicted by the multiple threshold 
hypothesis. Rather, the points follow the 
prediction of a Gaussian function, with 
some variability which too would be 
predicted with such small intensity dif- 
ferences as 2 per cent between steps. 

It is concluded, then, that the data of 
Experiments VIII and IX offer no sup- 
port for the concept of multiple thresh- 
olds as outlined by Motokawa. Responses 
to subthreshold intensities of current are 
indeed observed, but the percentage of 
positive responses appears to exhibit an 
approximately Gaussian relationship to 
the log of the stimulating current. 


SUMMARY OF RESULTS 


A brief summary of the results of 
these experiments is as follows: 

Experiments have been conducted in 
which the human eye was stimulated 
electrically by brief square-wave pulses. 
Three or four highly trained Ss were 
used in each experiment. The S judged 
the appearance or nonappearance of a 
phosphene in response to the pulse. A 
psychophysical method developed by 
Motokawa was used to determine the 
threshold for the electrical stimulation. 
This method involves successive stimulus 


* Attempts to obtain a constant stimulus 
function with 2 per cent steps, under similar 
light conditions, were made with two other Ss 
PMG and SHH. In both cases, however, the 
attempts were unsuccessful in that the range 
of intensities was not large enough to obtain 
an appropriate range of percentages seen. 


presentations in a descending series of 
intensities, as in the method of limits, 
followed by paired comparisons of blanks 
with successively weaker stimuli, When 
conducted under rigidly prescribed con- 
ditions this limits-comparison procedure 
can be used to measure changes in electri- 
cal sensitivity following flashes of light 
in the dark-adapted eye. More conven- 
tional psychophysical methods appear e 
be more reliable for determining electri- 
cal thresholds but are too time cit 
ing to be used in extensive series О 
threshold dominations. — 
Immediately following a flash ol € 
the eye is relatively inexcitable by — 
current, but within less than one аар 
after the light goes off the excitability 
rises to a normal or supranormal i 
Supranormal excitability, or spe 
ment, is not found in some 5% Ый hn 
clearly present in others. A study of va Я 
ous colors, areas, and intensities of a 
ulation has led us to conclude that t " 
degree of enhancement depends larga 
on the extent to which the light has sip 
ulated the rod, or scotopic receptor ше : 
anism, of the eye. The enhancement " а 
fect often persists for several sen 
after a flash of light. We have not pe 
any clear peak, however, in the pé 
course of such enhancement, 1 ih 
have not found the specific e is 
effects reported by Motokawa o ati? 
time course. Certain factors of €XP 
mental procedure are believed 10 
count for this discrepancy in 


anc 


Discussion 


i 
The principal purpose of these exp 
ments has been to evaluate the дё, sa 
of electrical stimulation of the €Y€ ub 
tool for studying the mechanism °? jur 
man color vision. After a general pe jca 

tion of methods of determining elect” 


- prima” 
thresholds (Experiment I), out pu 
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concern was to attempt to verify the re- 
sults reported by Motokawa for the en- 
hancement of electrical excitability by 
flashes of lights of various wave lengths. 
Later we were led to a new series of ex- 
periments in which we explored the role 
of the scotopic or rod response mecha- 
nism in the enhancement of electrical 
sensitivity. 


Comparison With the Motokawa 
Experiments 

Experiments II, III, and IV were de- 
signed to follow closely the procedures 
reported by Motokawa. Red, green, blue, 
and white lights were used with central 
fixation on a small field with luminances 
within the range that Motokawa had re- 
ported as effective. Duration of the light 
flash was two seconds, the value most 
often used by him, and delay times of 
1, 2, and g sec. were interposed between 
the flash and the 0.1-sec. square-wave test 
pulse of direct current through the eye. 
Three or four carefully trained Ss were 
used and adequate periods of dark adap- 
tation and rest were provided. 

Threshold determinations were made 
According to the design developed by 
Motokawa, A descending method of 
limits was followed by a comparing pro- 
cedure in which the final threshold was 
ascertained by the S's failure to distin- 
8'ish blanks from electrical pulses. De- 
Spite these similarities, we have not 
Claimed that Motokawa's experiments 
have been exactly duplicated in our lab- 
Oratory. The principal difference between 
Our experiments and his appears to Бе 
In. the application of the procedure in 
the daily routine of threshold determina- 
tions, 
. With such a high degree of similarity 
11 experimental conditions W€ should 
Certainly expect our own results to Te- 
Semble those of Motokawa in all im- 


portant respects. Nevertheless the extent 
of such agreement has been very limited, 
and may be described briefly as follows: 
First, we have indeed found some Ss who 
show impressive amounts of enhancement 
of electrical sensitivity resulting from 
flashes of light. The highest degree of 
enhancement seen under any of our 
conditions was one of about 0.4 log unit. 
This represents a drop in electrical 
threshold to approximately 4o per cent 
of its original value. In most cases, how- 
ever, much smaller values of enhance- 
ment were obtained. In fact, nearly half 
of our Ss failed to show any consistent 
enhancement effects under any condi- 
tions. We have not found any explana- 
tion for these wide individual variations. 
Second, we have confirmed rather closely 
the early time course of enhancement, 
reported by M otokawa and his co-workers 
for the first halfsecond after the light 
goes off. Third, we have found, in some 
Ss, a clear relationship between the 
amount of enhancement and the intens- 
ity of the light flash that produced it. 
While we have confirmed the existence 
of enhancement of electrical sensitivity 
by light, we have not been able to make 
use of this phenomenon as à tool for the 
study of color vision. This is because we 
have not obtained any differences among 
the various colors of light with regard 
to their aftereffects on electrical sensitiv- 
ity of the eye. Motokawa’s impressive 
series of studies, purporting to give an 
exact description of the fundamental 
response Curves for human color vision, is 
based upon these specific aftereffects of 
various colors. Motokawa’s consistency 
jn obtaining these effects is in marked 
contrast to our own negative results, ob- 


tained under a variety of experimental 


conditions. 1 
Differences in the routine of thresh- 


old determinations. We have corre- 
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sponded with Professor Motokawa about 
our negative findings with respect to 
specific wave-length effects. His opinion 
is that procedural differences are im- 
portant in accounting for the differences 
in results obtained in the two labora- 
tories. Motokawa has sent us sample 
protocols from one of his experiments, in 
order to illustrate the differences between 
his routine procedure and ours. With 
Motokawa’s kind permission, we have 
reproduced one of these protocols as 
Table 12. We believe that it contains a 
more detailed description of his proce- 
dure than has appeared previously. 


The protocol describes an experiment in 
which enhancement was measured following a 
flash of red light at various delay times, The 
typical time course was found, with the greatest 
enhancement occurring at a delay time of one 
second. The 5 was one of Motokawa's highly 
trained observers. We have taken the liberty of 
presenting Motokawa's protocol in such a wa y 
that the vertical position of cach entry corrc- 
sponds to the intensity of stimulation. The in- 
tensity values of the presentations are indicated 
on the left of the table, in units of electrical тс- 
sistance in ohms. As the number of ohms 
increases, the stimulating voltage decreases. 
Spaced out in this way, the values clearly indi- 
cate the differences in step size which Moto- 
kawa has used in arriving at his thresholds. The 
columns headed Ros, R, Ra, and R, contain 
threshold determinations at delay times of o. 
1, 2, and 3 sec. after the cessation of a 2- 
flash of red light. The columns headed R, in- 
dicate reference threshold determinations in 
which no light was present. The order of the 
columns indicates the order of threshold deter- 
minations in this experiment. Within cach 
column are notations of the Ss’ responses. An 
о in the body of the table is used to indicate 
that a phosphene was seen by the S, and an x 
indicates that it was not seen. The letters w 
and vw indicate the S's Teport that the phos- 
phene was weak or very weak. The notation 
com indicates the intensity at which the com- 
parison stage of the procedure was demanded 
by the S. At all subsequent intensities, the com- 
parison procedure was used, After the threshold 
was reached, several further comparison pairs 


? Motokawa's circuit for electrical stimulation 
can be found in reference 18. 
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were presented, in order to censure that = 
threshold obtained was the "true" one, and n 
an apparent onc. This is indicated by a Bow 4 
x: and all further single xs indicate that the 
S's response was "I don't. know." 


‘The concept of multiple thresholds, gs 
outlined in a previous section of шн 
paper, has led Motokawa to use gross 
steps in the initial phase of the deter- 
mination of a threshold. Furthermore, 11 
the determinations in which light was 
used, he employed gross steps in order e 
skip over the "dark" thresholds, Les - 
intensity values which had yields 
thresholds with no light present. Тїз 
procedure, also described by Macha? 
in a recent publication (22), was p 
gated to avoid apparent thresholds pe 
to ensure obtaining the true thresholo, 

In Experiments VIII and IX, peer 
consequences of the multiple rp 
concept were tested, "The weight of pis 
evidence appeared not to support sum 
concept, but instead to be n is 
with the usual view that a threshold ш 
a point statistically defined Ш а 
transition zone of stimulus цыш. 
over which the probability of a gn " 
response changes gradually from low 
high. 

The effect of slep size on à th 
Conventional treatments of psycho] 
make no provision for multiple d 
olds, assuming instead that à j 
threshold can be found by the ge Р 
equal steps along a continuum of ye 
ulus intensities. In a descending um 
there is no exact point at which tl 
changes his judgment from bunt 
“not seen.” Rather there is a trans!” 
zone from suprathreshold in 
Which judgments of "seen" predom 
to subthreshold regions in which an 
ments of “not seen" become more ў | 
more numerous. The threshold 18 Sint 
narily defined statistically as that 
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TABLE 12 
ADAPTATION OF A PROTO! ч К. 
ADAPTS N OF ; со ву MOTOKAWA (PRIVATE COMMUNICATION) SHOWING THE 
Ror TINE rog DETERMINING ELECTRICAL THRESHOLDS. THRESHOLDS ARE INDICATED BY 
HE POSITION or THE HORIZONTAL LINE NEAR THE Bottom or EACH Data COLUMN. 
(See text for details) 
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lus value for which judgments of 


and “ 
Not seen” have an equa 


о 
f Occurrence, 
he method of limits is one in 


1 probability 


“seen” the threshold is 


which 


rather differently de- 


fined. As Guilford (9) has pointed out, 
much depends on the 
judgment of each series, 


single terminal 
at which the S's 
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judgment shifts from “seen” to “nat 
seen” or vice versa. The limits-compari- 
son method of Motokawa has the ad- 
vantage that it removes some of this 
emphasis on single judgments by provid- 
ing that each series end only after the $ 
has tried unsuccessfully to make several 
comparative judgments between the 
stimulus and a blank. There is still a 
considerable transition zone, however, 
within which this criterion of unsuccess- 
ful responses can be met; and the prob- 
ability that it will be met at any time 
during a descending series increases as 
the stimulus intensity is reduced. It is 
also obvious that the number of unsuc- 
cessful responses is determined, at any 
point within the transition zone, by the 
total number of stimuli presented at that 
point. The use of gross steps entails the 
presentation of fewer stimuli within a 
given region, while finer steps increase 
the number presented. Therefore, in a 
descending method such as this, the 
threshold criterion of unsuccessful re- 
sponses is most likely to be satisfied in a 
region of the transition zone in which 
fine steps are used.15 

It is our contention that, in the de- 
termination of electrical thresholds, the 
probability of finding a threshold at a 
given point is increased by the use of 
fine steps in that region of stimulus in- 
tensities. Motokawa has realized this in 
his practice of using gross steps for 


"skipping over" the unwanted dark and 


apparent thresholds in his search for the 


"Guilford (9) has discussed a somewhat 
analogous situation in the determination of a 
difference limen. He says, "Having a precon- 
ceived notion that Weber's law holds, E will use 
smaller steps in the lower part of the scale and 
larger steps in the upper part of the scale, It is 
conceivable that, with a highly experienced O or 
E, unless measures are taken 
Weber's law or any other law co; 
strated to hold merely 
steps." 


to prevent it, 
uld be demon- 
by a systematic choice of 
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true threshold. He has used fine steps 
systematically in the region where the 
true threshold is expected to be found. 
It is evident from the protoc ol in Table 
12 that an elastic procedure has — 
used to select the size of step intervi" 
from one part of the session to another. 
In the limits stage of the determination 
Motokawa's routine calls for pee 
different steps of intensity for the “dar А 
series and the series with light. He S 
also varied these steps for the differen 
delay times used within the “light” s 
After comparing is begun, step pier 
of 50, 100, or 200 ohms have been as 
arbitrarily from one time to another. 
We believe that the effect of uds ed 
cedure is to heighten the probability ge 
large values of enhancement will чї 
shown for those particular ши, ^" 
which gross steps have been per 
relatively low intensity levels. A ius ple 
effect is that of yielding the rem: sae С 
degree of consistency that the i um 
thresholds have shown in the re to 
laboratory. We have never been ан зе 
achieve enhancement values confor™ үп 
to such smooth functions as are sho! 
in Motokawa’s papers. | 
Further confirmation of the im] 
of step size is provided by an exP tory: 
conducted in Motokawa's ios ' 
Through the kindness of Mote " of 
we have illustrated the results ol Е this 
his experiments in Table 13- vtm 
special experiment the scries de а? 
ing stimulus intensities has o Agait 
approximately equal log intervals. “ales 
the first column lists the resistance `‘ git 
in ohms, used to control the эти 
tensity. As in Table 12, ascending nding 
of resistance correspond to desce 
values of stimulating voltage- > indi 
The results shown in Table !? nd ? 
cate that Motokawa, too, fails ® "m 
specific crest time when equal 108 
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TABLE 13 


ADAPTATION OF A PROTOCOL BY MOTOKAW. 
X PERIMENT IN WHICH EQUAL Loc STEPS WERE EMPLOYED 


SPECIAL 
(See text 
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of intensity are used. That is, thresholds 


Obtained in the dark (Ro) condition аге 
No higher than those following flashes of 
ie light by one (R,) or two (R:) seconds. 
Consequently Motokawa concludes, and 
wg agree, that no specific enhancement is 
revealed by red light with a one-second 
delay time under the conditions repre 
а. by Table 13. Motokawa has in- 
terpreted this to mean that the method 
oF equal log steps involves t00 many 
Presentations of electrical stimuli above 


the threshold, with a resulting long-term 
insensitivity to the very weak stimuli that 
lead to such low thresholds as that of Ft, 
in Table 12. He has supported this inter- 
pretation by citing his most recent experi- 
ments (personal communication) which 
show that repeated high-intensity electri- 
cal stimuli produce long-lasting eleva- 


s in electrical threshold. 
ation is simply that 


mena appear only 
kipped over and 


tion 
Our own interpret 


the crest-time pheno 
when higher levels are s 
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stimuli are massed, by the use of small 
steps, at the lower levels. No long-lasting 
changes in electrical threshold have ap- 
peared in our own experiments. Such 
effects are negated by the following con- 
siderations: (a) Polarization effects or 
other changes in skin-electrode resistance 
are minimized by our use of the 200,000- 
ohm resistance in series with the S. (b) 
The data of Experiment I do not reveal 
a rising trend with repeated stimulation. 
(c) In our control experiment, thresholds 
obtained with a 7.65-sec. cycle were not 
higher than thresholds obtained with a 
15.3-5ес. cycle. 


The Achromatic Nature of Our 
Enhancement Effects 


Such enhancement effects as we have 
been able to observe appear to originate 
not from the color response mechanisms 
but from the scotopic system of the eye. 
This was first apparent in the course of 
Experiments II, III, and IV in which 
small central fields of light were used. 
It was under precisely these conditions 
that Motokawa obtained his most clear- 
cut results on the relationship of the 
time-course of enhancement to the wave 
length of the stimulating light. Experi- 
ments V, VI, and VII showed that even 
more marked enhancement effects oc- 


curred when the field of light was ex- 


tended to cover a wide retinal area. In 


this case the scotopic nature of the effect 
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was even more readily apparent. It was 
concluded that in all of these cases the 
degree of enhancement was теней te 
the extent to which the rod-receptor 
mechanisms of the eye were affected by 
the light. It was assumed that p 
light acting on peripheral receptors S 
counted for these effects under the sma 
central area condition of stimulation. | 
It is our conviction that the specific 
wave-length effects that Motokawa - 
described are the result of his very spe i 
method of experimenting in which pP 
protocol is modified in accordance e 
the responses of the S. We have "20 
unwilling to depart from à rigidly ed 
fined protocol in our threshold de A 
minations, fearing that any such p 
ture might influence the results 1n beo 
of some preconceived hypothesis. Ме dis 
ready to admit that our negative ue 
with regard to specific wave-length p 
may simply mean that these effects e 
so subtle that they are masked к p 
ordinary variability of our threshold » 
terminations. If this is so, icq ME 
can hardly endorse the usc of this mer Ў 
for studying the basic characteristics 
color vision. 


the 
" Consistent with this view is the fact е i 
phosphenes themselves appear айе, 
Suggests that the electrical current do 

affect the color response mechanisms 
tensities of stimulation are low. It 15 gphet 5 
true that Ss typically report that the aed field» 
appear in the periphery of the Ме found: 
where relatively few color receptors а! 


ў 
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A Factor-Analytic Study of Planning Abilities 


R. M. BERGER," J. P. GUILFORD, AND P. R. CHRISTENSEN! 
University of Southern California 


1. INTRODUCTION 


Te stupy is the fifth in a series of in- 
bil vestigations designed to explore 

abilities А : А а 

a ities considered to be important 1n 
lg ences Сү 

| successful performance of high-level 
эегес 121; ‘ 8 QE" 

nel? For the first four studies in 

Now ats 

бату S at System Development Corporation (for- 

баты, ystem Development Division of the RAND 
^?rporation), 


of ieee AMT N6onr 
paper 2M ж, The ideas expre 
resent the a ор and do not neces 
search, Re е of the Oflice of Nava 
Mitted ЙО, uction in whole or in part is per- 
ernment, ог a PRE of the United States Gov- 
ford is y hg i of this paper, J. P. Guil- 
the projec р Е le investigator and director ОЁ 
and R, M. Бере Christensen is assistant director, 
particular ds D Was in di charge of this 
Study see refer ie a fuller discussion of this 
P. R, Me “rences no. 2, 4, and 5. 

ment of "baie participated in the develop- 

dering hypothesis and contributed 


Breatly t 

а y to the ы А 

Vising of due construction of items and the de- 
We 1g methods, 

the 


are indebte 
(ү oe to the following persons їй 
ments indicated i Southern California depart- 
if pelicans a e e laa in 
Stance Loye Ребтепгаї testing: Dr Con- 
баа MU Psychology: Dr ae tr 
Jones ну D. W. Lefever анй Mr pas 
0 È e ü Mr. R. A. 
fully Prim Education, In addition we grate- 
adaptation by ie permission for test use and/or 
У the following authors and publish- 


exss р 
rs: R, I, Fhornd 


the Psy * ike (test 5); 
Hump 01081са! st 5); R. B. Cattell and 


Corporation | (test в); L. С. 


үр ЭЗ, 34. 35); D. C. Adkins 
lest o 


n 
чыз. to the staff of the 
91 “Чогу, HRRC, Lack 


demanding 5 £ 
g test battery, y of a long and 
“TUcular we wish to 


the series see (3, 6, 10, 13). The purpose 
of this study was to isolate and define the 
abilities involved in planning perform- 
ances, We were especially interested in 
planning as it enters into the activities 
of supervisory, military, and research per- 
sonnel. It is in the planning perform- 
ances of these groups that we have looked 
for clues as to hypotheses of ability re- 

uirements that are common in almost 
any kind of planning regardless of vo- 


cational context. 

There is little psychological literature bearing 
directly on planning; the most useful were the 
reports of previous factor analyses. A planning 
factor Was found in the AAF Aviation Psy- 
chology Research Program but was not clear- 
cut as a very general or fundamental ability of 
planning or foresight (9). Analyses were made 
of two special AAF batteries of foresight-and- 

poth of which included a small 


planning tests, 
Кера: of planning tests plus reference tests. 
The planning tests helped to define two new 


factors: а planning factor and an integration 
factor The agreement between the two analyses 
wis not very strong with respect to the new fac- 


tors. 
Two alternat 


Report No. 5 of the 
Research Reports (9) 


ive hypotheses were advanced in 
AAF Psychology Program 
to explain the nature of 


the planning factor. One hypothesis к: that 
it might represent а type of ideationa ее, 
Ке vh the idea that the man who RT da 
oti i i ime wou avi 
of more solutions рег Ч me wo 


is ex- 
an advantage with some of these ae = 
s T 
planation does І eli, 


not hold very W 
i important for 
roduction of id re impo: 


no mo: 
t were loaded on 


erformance on he а n that did not E 
the factor than for 0 d pellen, hypothesis is 


significant loadings. 


Director of Re- 
ical Aide, an 


personnel w 


the military 
proctors. 


istrators and 
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that some form of visualization different from 
the manipulation type is required in 5E of 
planning. Convincing evidence for this hypothe- 
is i o lacking. 

Ш = апаа of the AAF Sheppard Field 
battery of tests, Guilford, Fruchter, and Zim- 
merman (7) found a planning factor, which they 
labeled “planning speed." Most of the tests that 
appeared loaded on this factor had been analyzed 
in AAF studies discussed above, Due to the in- 
clusion of a large number of planning tests, the 
investigators felt the planning speed factor was 
probably more stable than the planning factor 
found in the previous AAF analyses. The new 
factor was termed planning speed, since the 
emphasis had shifted to tests in which speed 
plays a more important role. 

Lucas and French included several of the 
AAF planning and integration tests in a recent 
study (11). An integration factor corresponding 
to the AAF integration III was tentatively identi- 
fied, although the planning tests did not show 
up on the factor as they did in AAF analyses. 
The presence of two planning tests on another 
factor suggested it might be a planning factor. 
The evidence is not clear, however, since the 
loadings and communalities tend to be very low. 


II. HYPOTHESES 


Planning was conceived not as a uni- 
tary psychological function but as a term 
to cover certain classes of activities, On 
the basis of our considerations of plan- 
ning activities and previous studies, six 
major hypotheses were developed con- 
cerning the abilities involved. Four were 


formulated in terms of classes of activi- 


ties involved in the various stages of 


planning. These were: orientation, hy- 
pothesized as an ability to see an order or 
trend in a mass of information and to 
recognize the pertinent variables operat- 
ing in a situation; prediction, the ability 
to see beyond the material given; elabo- 
ration, the ability to produce ideas or 
representations of ideas that contribute 
to the development of plans; and order- 
ing, the ability to arrange bits of infor- 
mation into a meaningful series, Two 
additional hypotheses were constructed 
pertaining to the qualitative aspects of 
planning. They were: ingenuity, the 


тар 5 
ability to find new and different method 
nh n 
of accomplishing a task; and — э н 
the ability to detect limitations 1n p e 
M dde aes D 
and to determine relative importance 
variables. " 
Each major hypothesis indicates an p 
: :Akine ability. Bac 
pected primary thinking ability. Ez 
3 ific atu 
subhypothesis reflects a specific fea e 
or variation or alternative ipo ue 
the primary-ability dimension. These eA 
cpg ay. [vi 
potheses апа subhypotheses have ser 
$ "ас recog- 
as the logical base for what was rec А 
H Е n + oars T ^ ra 
nized as a continuing research. prog "o 
in which this study is an exploratory E 
оа ation: oi 
vestigation. From an operational р E 
of view, the subhypotheses served as sta 
ing points for test ideas. aibi 
The complete list of hypotheses, ке 
hypotheses, and the tests (by number) 
signed to measure them follows: 
1. Orientation т 
жой ; er 0 
а. Sensitivity to order: seeing an ӨЕ оп 
trend in a relatively ambiguous SI“ 
(tests 1-3, see Section III). 
b. Recognition of variables ane 
pertinent variables operating in а 5! 
(tests 4-5). 
. Prediction ae 
< of a SP? 
a. Extrapolation: seeing the effects of Lent 
tial or temporal extension of a P 
trend (tests 6-8). | 
b. Foresight: relating what is giv 


lies ahead (tests g-12). 

3. Elaboration sary de- 
а. Specification: specifying the neces 

tails (tests 13-15). . finding 

. Production of alternate methods: ateria 

different arrangements of the same р ап 
that will fulfill the requirement 

adequate solution (tests 16-17). 


the 
awareness of ! 
awar tion 


rj 


en to what 


b 


рте" 
с. Symbolization: producing adequate " 
sentations of ideas (tests 18-19)- 
4- Ordering ps in á 
a. Temporal ordering: arranging step 
time sequence (tests 20-22). " ject 
b. Hierarchical ordering: arranging ss (1658 
matter according to class and subcla 
23-25). 
5. Ingenuity mmo? 
a. 


New methods; finding new ог ine 
procedures (tests 26-27). js to Пё 
New applications: adapting metho“ 
Situations (test 28). 


b. 
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6. Evaluation 
ы Importance of variables: determining rela- 
tive importance of variables (tests 29-30). 
b. Seeing deficiencies: detecting the shortcom- 
ings of a proposed procedure (test 31). 


Reference factors (previously known 
factors expected) ате: planning (AAF 
analyses), integration III, verbal com- 
prehension, general reasoning, logical 
evaluation, originality, ideational flu- 
ency, associational fluency, adaptive flexi- 
bility, eduction of conceptual relations, 
judgment, visualization, symbol manip- 
ulation, and numerical facility. 

Included in the battery is a test that is 
Necessarily complex in that it requires 
the examinees to devise a complete plan. 
ү test is an adaptation of Lorge’s 

lanning Skills test. A detailed account is 
Blven of a military situation in which 
there are problems of morale and leader- 
ship; the examinee is required to write 
doi the steps in a plan of his own mak- 
Ing that will improve the situation. Fur- 
in description of the test will be given 
to нао III. It should be informative 
of € to what extent a score on a test 
general planning will load on the 


Various ú : 
si lous factors that emerge in the analy- 
Я 


III. Tue Test BATTERY 
gree considerations influenced the 
ї А 
of 105 of new tests constructed and kinds 
Scores used: i 


1. Man 


p i ion-type 
answers, у of the tests require completi! ур! 


activities It was felt that since some planning 
Of the abt necessarily productive, the шеш s 
the basis ities associated would best be made 0 

Án of productive efforts of the examinee. 
Which ет. was made to develop т 
һе Scc опен of the completion type: mig 
Or tho: objectively. A minimum requiremen 
M Se to be scored subjectively was that it 
cor, Possible to develop a reasonably reliable 
where’ Parately timed parts were constructed 
caning ble to permit the computation ^ 
4A Bful alternate-forms reliability estime 
n effort was made to construct unique 


tests where possible, that is, tests of one common 
factor, except in a few cases where the task pre- 
sented is of necessity complex. 

5. Where possible, simple tasks and situations 
were used in order to minimize variances due 
to differences in personal experience. 

Most of the new or adapted tests that 
were to serve as experimental tests were 
pretested on groups of college students in 
order to determine adequate lengths. 
time limits, and instructions. Informa- 
tion derived from this pretesting was 
used in rejecting some of the tests and 
in revising and improving the 51 experi- 
mental tests to be included in the final 
battery along with 16 reference tests. In 
addition, several Air Force test scores 
from the classification battery were in- 
cluded in the analysis, also for reference. 

АП tests included in the analysis are 
described in the next section. For con- 
venience, they are arranged in the order 
of the hypotheses rather than in the order 
of administration. For each test, the na- 
ture of the task is stated briefly, and 
wherever possible a sample item is given. 

It should be noted that the sample 
items are transparently easy, since they 
were used as part of the test instructions 
for orientation. With respect to difficulty, 
then, the sample items are not typical of 
the items in the test proper. 

The number of separately timed parts, 
the number of items per part, and the 
working time are indicated. Since the 
scoring procedure for many of the tests, 
especially those requiring subjective eval- 
uations of completion responses, may be 
of interest, а summary of the scoring pro- 
tocol is also given for most of such tests. 

The code numbers of the experimental 
tests and some reference tests developed 
at this laboratory indicate the following 
information: The first letter—P, R, C, or 
E-indicates planning, reasoning, creativ- 
or evaluation, respectively. The sec- 


ity : 
* ds for a specific hypothe- 


ond letter stan 


R. М. 
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sis; in the case of planning, R means ori- 
entation, X means prediction, E means 
elaboration, O means ordering, I means 
ingenuity, and J means evaluation. Next 
comes the number of the test under the 
hypothesis, usually in order of the sub- 
hypothesis listed: 01, o2, etc. The last 
letter indicates the form of the test, A 
meaning the first form, B the second 
form, etc. Several experimental tests are 
uncoded; these are tests adopted or only 
slightly adapted from other sources. 


А. DESCRIPTION OF EXPERIMENTAL TESTS 


1. Orientation 
a. Sensitivity to Order 


Test т. Matrix Отает-РКОТА. Each 


А. mouse rat lion pig 
Answer: Animals become larger. 
B. century year decade 


Answer: Time unit becomes smaller. 


item is a 3 X 3 matrix of words. E (the 
examinee) is required to draw a straight 
line through the sequence of three words 
that are in the most meaningful order. 
The line may be drawn through any one 
of three columns, any one of three rows, 
or either of two diagonals. In addition; 
the direction of the order is indicated by 
arrowheads. An arrowhead at one end 
shows the temporal order of the sequence 
and arrowheads at both ends of the line 
signify a spatial or logical relationship 
that can go either way (see Fig. 1), 


Sample items: 


А. run think want B. 


read write sleep accident 


walk publish book mixmaster valley 


Fic, 1 


An order in time is found in sample A. 
Order in space is found in the diagonal 
of sample B. The double arrow in sample 


B indicates that the sequence “skin-shirt- 
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coat” is as logical an order as “coat-shirt- 
skin.” 

Scoring: ated i 
only when both the correct line is oe 
and the correct placement of the arrow- 
head is made. The total score is the nunt 
ber of items correctly marked. | 

Parts, 2; items per part, 15; working 
time, 18 minutes; maximum score (high- 
est possible соге), 30. e" 

Test 2. Seeing Trends RIOGOA. ра 
item consists of a group of words ag 
arrangement is not entirely uniform. м 
attempts to discover a trend from v 
right and writes his description of th 
trend below the item. 


An item is counted correct 


Sample items: 


Jephant 
cow horse ele} 


тше 
second Е 


Note that in both items several ds 
versals of the trend occur between pen 
taken as pairs and it is necessary LO ү 
beyond adjacent words for the predo” 
nant trend. 

Scoring: Total score is the number М 
correctly marked items. Only опе Ww" 
of response is keyed for each item tly 
some cases responses worded ce 
than the keyed ones are acceptable е аге 
meanings are close or if the responses ® 
negatively worded. .orking 

Parts, 2; items per part, 10: wor 
time, 10 minutes; maximum score, E A. 

Test 3. Sensitivity to Order—PRO™ | 
Each item consists of 5 words - «is 
ordered in some way. Some of the е 
present the words in the right pis 
others have one word out of order. u , s 
arrangement as given is correct, Ё n heir 
a plus sign. If the words are not in t the 
best order E is required to circle 


А ut 2 
word that is out of place and to P 
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check to show where that word belongs 
in the group (see Fig. 2). 


Sample itema 


A^ k 
» жу lock, 
E door, room. house _F 


B. aceid 
ent, 
(rites) reporter, newspaper, reader 


Fic. 2 


hs sample item Л the objects are in 
umi si Жы e d item B the ar- 
rime the = as good as it might be, 
нш А aie ofa Еренак 15 usually 
deni Pa A ore the Yeap of an acci- 
ind « eda made. \\ rite-up” is circled 
тат: indicates that it belongs be- 
Scant porter and newspaper. 

dinee D сау the items in which some 
ê ler A be indicated in the order 
ranged е ae presence of correctly ar- 
quired to s Ua the examinee is re- 
keep the pae with a "plus" serves to 
changes A — from indicating 
to be "s NS items that also appear 
items P кнн arranged to him. The 
Propriate wd scored must have the ap- 
correctly Mri Ж; апа the check 
The total — 0 order to receive credit. 
Parts, з, » the number right. 

time, 8 pine per part, 15; working 

es; maximum score, 18. 


b.m 8 
€cognition of Variables 


Е iat £ Pertinent Questions-PRO34- 
Situations ан with a number of conflict 
ion wu os additional informa- 
Choice, Æ js De required for making a 
Ры asked to write four questions, 
Pasis for cg to which would serve as à 


М aking a decision. 


ample item: 


A stu 
PR Spe Who has just graduated from college 
gı шу, oe in two different parts of the 
“dered in nat four questions have to be con- 
Ossiby Making a choice? 
be in rere: What would the work in- 
т. n an two jobs? What salaries would be 
tive o tat places are the jobs? What ате the 
Pportunities for the future? 


Other questions may be acceptable but 
they may not repeat previous questions 
and they must refer to different aspects 
of the problem. 

Scoring: One point is given for each 
response that refers to a separate aspect 
of the problem. The total score is the 
number of responses credited. 

Parts, 2; items per part, 4; working 
time, 12 minutes; maximum score, 32. 

Test 5. Awareness of Variables—PRO- 
44. This is a test developed by the 
Human Resources Research Laboratories 
at Bolling AFB. A problem is presented 
in which some decision is needed. Ac- 
companying the problem are a number 
of statements of facts that may be perti- 
nent to the problem. E's task is to select 
a specified number of facts that seem 
most important in reaching a decision. 


Sample item: 
You are driving from Middleton to Warrens- 
f time for the trip. 


burg. You have plenty о 
There are two possible routes: Route A and 


Route B. Which would be better to take? 

A. Route A is 105 miles long and Route B is 
115 miles long. 

в. Route A goes through 5 small cities on the 
way, while Route B goes through $ some- 
what larger cities. 

C. Route A has 10 mi 
tion, with dirt sur! 
way traffic. 

D. Route B is mostly 


les of road under construc- 
face and sections of one- 


winding, bumpy macadam, 


while Route А is mostly 4-strip concrete. 
. Route A goes north of Lake Winepaw, while 


Route B goes south of the lake. 

which two facts аге the most important to 
consider in deciding which route to take? Circle 
the letters preceding the two statements se- 
lected. 

The best answers 
to be C and D. 

In the test 1056 
facts under it, an 
facts that are most 1 

Scoring: Expert ор! 
determining the “most 
for each of the four pro 


= 


are generally agreed 


If, each problem has 15 
d Е is to select the five 
mportant. 

ion was used in 
important” facts 
blems; the facts 


6 R. M. BERGER, J. P. GUILFORD, AND P. R. CHRISTENSEN 


on which most experts agreed constituted 
the provisional key. Scores were obtained 
using this key and the results were sub- 
jected to item analysis. This led to cer- 
tain minor modifications resulting in the 
final key. The number of “important” 
facts in this final key varies from 4 to 5, 
depending on the empirical results. In 
scoring, a weight of 1 is given to each 
agreement with the key. The total score 
is the number of such agreements. 

Parts, 4; items per part, 1; elements 
per item, 15; working time, 12 minutes; 
maximum score, 18. 


2. Prediction 
a. Extrapolation 


Test 6. Series. This test is an adapta- 
tion from the items included in a part of 
R. B. Cattell's 4 Culture-Free Test. Each 
item consists of a series of figures, which 
are varied in some systematic fashion. E 
has to select the figure that goes next in 
the series by choosing among six given 
alternatives (see Fig. з). 


Sample item: 


D амор 


р 


MM 
Fic. 3 i i 


Scoring: Score is the number of items 
correctly marked. 

Parts, 2; items per part, Part 1-5, Part 
2-8; working time, 5% minutes; maxi- 
mum score, 15. 

Test 7. Effects—PXOrA. Given a state- 
ment of some present-day trend, E at- 
tempts to predict what the nature of the 
effects of this trend will be at a specified 
future time. Each item consists of the 
statement of a trend and provides room 
for writing down four possible effects, 
The trends given are realistic and cover 


many different areas in which there is 
common experience. 


Sample item: 


irls m i һе 
A. There have been more girls born in e 
last five years than boys, What effects will “ы 
have 20 years from now (in addition to EE 
í Y! d cf OM 
obvious effect that there will be more W 
than men)? 


Possible answers: More unmarried women Hain 
unmarried men. More rivalry among women ү 
husbands. More women would take up ees 
work. Agitation to allow polygamous marriage 


Scoring: Four responses can be given 
to each item and each 
weighted either o, 1, or 2. If the response 
is a duplication of a previously given ia 
or is not a possible effect of the tren? 
given, it is weighted zero. Effects that ш 
possible are weighted 1; ап ae 
weight of 1, or a total response weight р 
2, is allowed if the effect is less obvious 
or is somewhat indirect. The total score 
is the sum of the weights. 

Parts, 2; items per part, 5; 
time, 16 minutes; maximum score, us 

Test 8. Consequences (Remoteness)- 
CFO6A. For each item in this test ? 
change of an unusual nature is нарог, 
which would disrupt the normal m 
affairs. E is required to list as many pac 
ferent consequences or results of thes 
changes as he can. 


response 15 


working 


Sample item: 


al 
к i d loct 
What would happen if all national an 


laws were suddenly abolished? 
Possible answ People would no lor 


parking tichets. There would be more 
No more taxes. 


iger gel 
killing 


erived 


Scoring: There are two scores d à 


from this test: a low-quality score “г 
remoteness score. For the Том Ча) 
Score the number of direct r ae 
determined. Direct responses are ne 
that describe direct or immediate QU 
quences of the given changes. The ide 
quality score is used as a measure of ! te 
ational fluency (see test 44). The pec 
ness score is the one used here as а 16 


و 
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ur — а А 
и of prediction. The score is based on 
1 T it 
€ number of indirect or remote conse- 
qu OB Tr A Ж 
]uences listed. The individual responses 
are т; MEM 
; rated as direct or remote by compar- 
n > z М ; 
| пиц with а comprehensive list of 
Model c. = 
| E consequences that had been shown 
о bé rel: ; ; ; ; 
І Be reliably weighted in a previous 
scor AM "m ; 
" ing for the creativity analysis. Every 
SDOnse | S s b ы 
те Ponse is thus scored either 1 or o for 
me i е 
; dleness, "he total score is the sum 
of the weights, 
Par ss А 
z ts, qi items per part, 1; working 
time, § mi В = 
bs Minutes; maximum score, unre- 
Stricted (U). 


b. Fo resight 


Тең 9. Competitive Planning. This is 
Д е test modified slightly for 
on the pss m study. Ihe test is based 
game, Ëa иш completion-of- squares 

nents (Black bes чо hypothetical oppo- 
as many А . 0 W hite) tries to complete 
are HO Aang as possible in figures that 
or both Kd ete. E has to plan the moves 
the aa so that each achieves 
Sible, ac ch. number of squares pos- 
filling in always makes the first move, 
Square, p. he side of an incomplete 
Pletes 4 Pee time Black or White com- 
ditional et he has to make опе ad- 
many nove. E must look ahead > 
siy; 

“чә. E immediate. gain should be 
Sible Dec in favor of one making pos 

^ ater gains for future moves. 


an 


Cases i 
ases in order to see that a mov 


lis ds 4 
Answer is an answer-sheet test and an 
Ou legend provides five alternative 
сот 


Fig. 4). 
Answer Legend 


| A. Black 0 white 4 
B. Black 1 White 3 

Zn C. Black 2 White 2 
| D. Black 3 white 1 

E. Black 4 White 0 


Making the best moves possible for 
their individual interests, Black and 
White complete this sample problem in 
the following manner: as always, the first 
move is made by Black. No matter which 
side Black fills in, White is able to com- 
plete two squares immediately. After 
completing the second square, White fills 
in a side of one of the remaining squares. 
This enables Black to complete the re- 
maining squares so that the final result 
becomes two squares for Black and two 
squares for White. The answer marked, 
therefore, should be C. 

Scoring: The score is the number of 
items right minus the number wrong di- 

Ww 
vided by four (R ——). 
4 

Parts, 2; items per part, 20; working 
time, 20 minutes; maximum score, 40. 

Test 10. Symbol Grouping—PXO2A. 
Each item consists of a line of X's, —'s, 
and O's presented in a scrambled order. 
The task is to make successive moves of 
groups of one or more adjacent symbols 
to achieve the correct order. The desired 
r for all items is first all the X's, next 
and last all the O's. The score 
only on finding a solution 
but also on making as few moves as pos- 
sible. Since there are a variety of ways of 
getting tO the correct order, foresight is 
hypothesized to be demonstrated when 
the most economical procedure is 
adopted. E indicates each move by cir- 
cling the symbol or symbols to be moved 
and placing а checkmark at the т 
they are to 80: This is repeated for each 
new arrangement resulting until the so- 
lution is reached (see Fig. 5) : i 

Scoring: A weight of 2 is given w hen 
the solution is reached in the minimum 


number of steps 


and a weight of 1 is 
given for the minimum number of steps 
plus one more step. 


orde 
all the —5$, 
depends not 


Two steps or more 


GE GUILF! i TENSEN 
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Sample item: 


i ез lead to the 
i X - O X -, the following mov 
Given: X - O X 
correct order. 


хб охх-о(&Ә 

x x - o£ xo 

XXXX---O0 
Fic. 5 


beyond the minimum is felt to indicate 
little or no foresight and is therefore 
given no credit. Unsolved and un- 
attempted problems are also not credited. 
The total score is the sum of the weighted 
item scores. 

Parts, 2; items per part, 8; working 
time, 10 minutes; maximum score, 32. 

Test тт. Contingencies—PXO3A. In 
each item an object is named that is to 
be considered in connection with a given 
situation. E's task is to state a possible 
condition that might require the use of a 
given object in that situation. Four ob- 


jects are presented with each item in the 
test. 


Sample item: 


Sally and Jane are going out to a farm to pick 
berries for the day. Their pay will be half the 
berries they pick. Included in the things they 
will take are the objects listed below. For each 
object, state a condition or circumstance that 
may arise requiring the use of that object. (Do 
not merely give the use of the object, but write 
why it might come into use.) 

Ointment If bitten by insects or scratched. 

Pins For possible rips in the clothes. 


Scoring: Each written response is 
scored as either acceptable or unaccept- 
able. A response is acceptable if a plaus- 
ible condition is given that might arise 
and require the use of the object. The 


total score is the number of acceptable 
responses. 


Parts, 2; items per Part, 20; working 


time, 12 minutes; maximum score, 40. 
Test 12. Route Planning—Cl4rrAX. 


This is an adaptation of an Air Force 


test. For each item E must trace a path 
from the corner of a printed maze to a 


goal box in its center. ‘There are m 
item numbers, one at each corner ot se 
maze, that serve as starting points. T | 
task is to locate the one point — 
which one must pass in going to the goa н 
Letters mark the points оп the -— 
routes between the starting point poo 
the goal. When E finds а qe ie 
which he must pass through in ort er i 
reach the goal he marks the letter cw F 
special answer sheet provided (see Fig. 


Sample item 


Y 


Fic. 6 


In going from 91 to the goal, yov. ш 
pass through either A or C; HONEY | 
must pass through D; therefore, р em 
right answer. Answers for other ! 
are: 92-В, 93-G, and 94-р. 

Scoring: R — W/2. vorking 

Parts, 5; items per part, 8; ж 40. 
time, 17 minutes; maximum scores; 


3. Elaboration 
a. Specification 
п—РЕО?® 
ing 
is 


Test 13. Planning Skills 1ч 
"hie ш te ж adaptation О я 
This test is ап adapt atter 


i 
ae рое; 
conditions prevailing at some hyp it 


cal military, weather-station c 
the Arctic. The conditions for p rc 
part entail the effect of the rigors ° 1500" 
tic temperature and isolation on P 

nel assigned to the stations. a z 38 
the several paragraphs a pat 
given the signal to turn to the he write 
of the test, which requires that ^ 
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as detailed a plan as possible on some 
given aspect of the problem situation. 
а Же second half of the test another 
particular aspect of the problem situation 
E Sen and E is to write down all the 
details necessary for coping with that 
part of the problem. 

Scoring: Each detail listed, if specific 
enough, is given credit. The total score 
м in number of acceptable details. 
nr a» items per part, 1; working 

a ) minutes; maximum score, U. 
е, = Iq. Planning Elaboration—PEO- 

en ‘ach part consists of a statement in 
т кер, plan of activity is briefly out- 
details < тарі is to fill in as many of the 
plan Tes е nbus necessary to make the 
is orien ‚ Each point or detail specified 

vritten on a separate line. 
one The total score is the number 

У ails specified. 

i 2; items рег part, 1; working 
Tex ) minutes; maximum score, U. 

E Figure Production—PEO 34. 
df onet s m Ша test consists of markings 
given i. o1 another. The markings as 
task is = no particular meaning. E's 
ле feels cim what is given as much as 
and pince. to produce a complete 
ügure h gen figure. E can draw any 
Markin ee as long as the given 
What he; arg definitely incorporated in 

Makes (see Fig. 7). 


Sample item; 
Given; 
©з 


Possible answers; ET. 
. this LA 

S Fic. 7 
siones D. A total of 140 sample Te 
b thre © the stimuli were categorized 
terval ; raters into a -point equal- 
Sponge, scale of “elaborateness.” The re 
used ag on which all raters agreed were 
Bories с оа for their respective cate 
from | ategories were assigned weights 
© (least elaboration) to 4 (greatest 


elaboration). The remaining responses 
were weighted according to the category 
in which they were placed by a qualified 
rater. Score is the sum of the weights over 
all responses. 

Parts, 3; items per part, 4; working 
time, 9 minutes; maximum score, 48. 


b. Production of Alternate Methods 


Test 16. Alternate Methods—PEO44A. 
In each part a task is described that may 
be carried out in a number of nearly 
equally effective ways. E attempts to write 
down as many as six different ways of dis- 
tributing the work and assigning person- 
nel in order to accomplish the task. 


Sample item: 

A house located near a stream is on fire. 
cach carrying а bucket, arrive to 
help put out the fire. The house is about 20 
yards from the stream. In how many Ways could 
you organize this bucket brigade to deal with 


the fire? 

Possible answers: @. Form a line of 20 men; 
the first man fills a bucket and it is passed from 
man to man; the last man throws the water on 
the fire and runs back with the empty bucket. 

b. Form two lines of то men. The filled 
buckets are passed up one line and the empty 


ones back down the other. 
с. Assign 5 men to fill the buckets, 10 men to 


run with the buckets, and 5 men to throw the 
water where it is needed. 

Scoring: In order to be counted as an 
acceptable response, each method listed 
must be practicable, employ all or most 
of the given material, and be in some way 
different from the other methods pro- 
duced. The total score for the four items 
is the number of acceptable responses. 

Parts, 4; items per ра 1; working 
time, 14 minutes; maximum score, 24- 

Test 17- Match Problems II—PEO54A. 
This test is а varied form of the Match 
Problems test to be described in a later 
46). These tests utilize the 


section (test s t ; 
match-stick problems 1n which groups o 
form patterns of 


s are laid out to 
angles. E must find a way 


Twenty men, 


matche 
squares or tri 


10 R. M. 


of taking away a specified number of 
matches to leave a certain number of 
squares or triangles remaining. In test 17 
E has to find several different solutions 
for each problem. That is, different pat- 
terns of matches must be removed to 
leave the specified number of squares or 
triangles. Three or four different ways of 
solving each problem are possible (see 
Fig. 8). 


Sample item: 


TAKE AWAY 5 x X © KA 

MATCHES AND INN TAVAN /NIN AL 

"amus ДӘУ AN ДАА ЛАА 
Fic. 8 


The matches to be taken away are in- 
dicated by crossing them out with short 
heavy marks. In the sample item above 
the solution is reached by removing 5 


matches in the four different patterns in- 
dicated. 


Scoring: Credit is given for each cor- 
rect, different pattern. The total score is 


the number of acceptable solutions. 


Parts, 2; items per part, 5; working 


time, 20 minutes; maximum score, 98. 


c. Symbolization 


Test 18. Symbol Production—PEOGA. 
In this test E must produce symbols of 
his own invention to represent given ac- 
tivities and objects. In order to be accept- 
able the symbol produced must be a 
meaningful one for the action and the 
thing presented (see Fig. 9). 


E is instructed not to draw stick fig- 
ures or cartoon figures performing the 
Sample item 
Symbolize the underlined words: 
Ring the bell 
[I (2) 


Open the door 
(3) (4%) 


Look into the room 
(5) (6) 


Close the window 
7) (в) 
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acts but to try to symbolize the actions 
themselves. . Gens 
Scoring: Obviously а wide variety А 
symbols may be produced in this ш, - 
response is judged as an шерине s 
bol if some relation, however remote 
can be seen between it and the object ЕЯ 
action to be symbolized. Cartoon. pata 
are not acceptable unless there 15 nS a 
tional material that can stand alone < 4 
proper symbol. The same symbol ы 
for different items is scored only on 1 
first occurrence. The total score is the 
number of acceptable symbols. -— 
Parts, 2; items per part, Part 1-31» aN, 
2-30; working time, 10 minutes; ma? 
mum score, 61. Рес 
Test 19. Line Drawing—PEO74. + 
is a test developed by J. P. and Жи 
Guilford as a prognostic test of pigro 
in design (7). E is given a list ol oo ч 
tives for each of which he is to jun М 
simple line that expresses the meaning ня 
the adjective. Any kind of line тау re 
drawn, such as straight, curved, in pn 
form of a wave, or an angular patte 
The test is not a speeded опе; 
is allowed to finish. forms 
Scoring: Each line is scored for ie 
direction, and type. Lines are desc 
аз to form in four ways: Wave поп iS 
angle, and straight line. The d€“ (пр 
scored in three ways: horizontal, ae! 
ward slant), and down (downward E y 
The type may be heavy, mediul a 
light. This classification gives ш, 1ine 
gories within which to classify кат e fot 
in the test, i.e., four for form, шге the 
direction, and three for type. bun 
responses from half of the ni сас 
group, the frequency with which 


wa 
d : ries 
line fell into each of the 10 catego 5 


га 
tabulated. The frequencies ipe: аі ov 
lated into sigma values above anc iby? 
the mean and these values, adjust P E 
shift of zero point to make Баша ré 


Ct 
= P The 5 
tive, were used as the weights. 


everyone 


curve 


—— i 


А FACTOR-/ 


for each line is the sum of the weights 
for form, direction, and type. The total 
oe is the sum of the line scores. 

Pars, 1; items per part, 24; working 
ume, 10 minutes; maximum score, 242. 


4. Ordering 

а. Тетрота1 Ordering 
i Test 20. Temporal Ordering—POO14. 
У; ний E is given a short description 
НА i i ines а list of some of the steps 
ане Бе in the project. The steps are not 
Dur ile At pis order as listed. Following 

^ USt are questions that concern the 
UM of the steps in different 
by fillin ше list. E indicates the answers 
thea 8 us ins letters corresponding to 

Appropriate steps. 

Sample item: 
Fixing а Flat Tire: 
А, Mig gou are driving along a busy highway 
т А night, your right rear tire goes flat. 


The $ 

steps descr 

be cps described below are some of those 10 
performed, 


+ Set 
3 x P е 
со ut flares that had been stored in glove 
compartment, 

i " 
ái Bhten nuts on wheel. 

Ock whe 


} s so car won't roll. 
е Rej. саг with bumper-jack. 
f. ОШ ace flat with spar ч 

ock 
g Жш the trunk to get wheel blocks. 

ча. эипрег-]асК out of trunk. 
th ank ; 
е blanks left in the following statements, 


enter 
the Tetter. indic ч 
ог Steps letters indicating the appropriate step 


1. TE 
le: атны | | 

2 8 ^r that should precede step d include: 

gy at Ty Be 
2. The pa 
irs —À А | | 

а, t two steps of this plan, in order, аге: 


* The tas 
last step of this plan should be step b. 


њр The examinee had to decide, 
Which foll which elements preceded and 
Ea n owed the reference elements 
4, Particular question. A weight of 
Placed, | dina each element properly 
Which th 9 correct for “padding,” against 
Weight «t examinees were forewarned, a 
Correct] —1 is given each element 10- 
Was re Y placed, Where the proper order 
quested, bonus points are added 


~ 
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for responses in which the elements 
within a response are correctly ordered; 
points are deducted if the examinee’s 
order is too far from the correct se- 
quence. The score is the sum of the 
weights over all elements of the re- 
sponses. 

Parts, 4; items per part, 8; working 
time, 16 minutes; maximum score 75. 

Test 21. Picture Arrangement. This is 
Dorothy C. Adkins’ test based on se- 
quences from the comic strip known as 
"Louie," which has been adapted with 
the permission of Press Features, Inc. 
Each item consists of a separate sequence 
in which the pictures are disarranged. 
E is required to number the pictures 
to produce what he considers the most 
sensible order (see Fig. 10). 


Sample item: 


— Пре ||| 


| 


Fic. 10 


the pictures should be 


To show how 
ake sense, a “1 


arranged in order to m 
s been written under the picture that 
лоша come first, а “ə” under the pic- 
ome second, and so on. 


al score is the number 
of correctly ordered sequences, i.e., items. 

Parts, 2; items per part, 1-8, II-12; 
7 minutes maximum 


» 


ha 
sl 
ture that should c 

Scoring: The tot 


working time, 
score, 20. | 

Test 22. Sentence Order—RPO4B. This 
was adapted from a test in Dorothy C. 
Adkins’ North Carolina battery of yé- 
soning tests. Each item consists of three 
sentences, which are to be arranged in a 
sensible order. E indicates the order by 


numbering the sentences. 


Sample item: 
2 She bought some food 
She returned home ап! 

food she had bought. 
1 She went to the market. 


at the market. 
d cooked some of the 


2 R. M. BERGER, J. P. GUILFORD, AND P. R. CHRISTENS 
1 . А 


Scoring: The total score is the number 
of items in which the sentences are cor- 
rectly ordered. | 

Parts, 2; items per part, 10; working 
time, 6 minutes; maximum score, 20, 


b. Hierarchical Ordering 


Test 23. Outlining I-POO2A. In this 
test a series of statements of differing 
degrees of generality are given and E's 
task is to arrange the given statements 
so that the more particular (less general) 
Statements are grouped under the more 
general statements with which they have 
most in common. One statement, the 


most general of all, is to be designated 
as the title. 


Sample item: 


4. Cats are useful as mousers, 
B. Dogs are fine as guardians, 
С. Cows give milk, 

D. Animals are useful 


Е. Some pets are usefu 
able. 


F. Sheep furnish wool, 
С. Farm animals are essential to our Way of life. 


in a variety of Ways. 
1 as well as being enjoy- 


Instructions: Put the letter for each of the 
statements above in one of the Spaces below so 
аз to make the best outline, Use all statements; 
use each statement only once, 


Title D 
LG 


"10 


IL E 


ым 


In the test Proper the context of the 
hierarchical ordering task involves the 
characteristics of eggs as а food. 
Scoring: Weights are assigned to each 
element on the basis of its being grouped 
with the other elements with which it 
Properly belongs and on the basis of its 


being at the Proper level of generality 
as intended by the co 


item. Score is the sum 
elements of the response, 


N 


Parts, 1; elements per part, 16; Жой 
ing time, 5 minutes; maximum score, 
2i ВЕС 
Test 24. Outlining П—РООзА. This 
test was administered as Part П of the 
Outlining Test, for which the dese rage 
appears above (test 23, Outlining I). T 5 
context of the hierarchical ordering tas 
is the planning of a club meeting. 

Scoring: See test 23. рат. 

Parts, 1; elements рег part, 16; р. 
ing time, 5 minutes; maximum score, dn 

Test 25. Word Matrices-CYOs4. in 
this test E is given several word pui 
each of which is composed of three до 
zontal spaces and three vertical л 
Two or three of the spaces have ae : 
been filled in with words which "i d 
pattern for filling in the rest eR rds 
spaces. E’s task is to arrange the ше. 
given below each square in their ы! J 
places so that the words of each ande 
the square will show the same rela 
ships. 


Sample item: 


4 net 
minnow = 
Е rod 


The proper arrangement is: 
net 
rod 
harpoon 


minnow 
bass 
whale 


pool 
lake 
ocean 


-onta 
i orizo! 
he pattern in each of the h Л 


is : jt is ust i 
lines is: type of fish, place it "s it 3 
found, and instrument with whi j 


15 
spica Jum! 
caught. Each of the vertical co ac 


ol. 
lake, whale, ocean, bass, harpoon, po 


ig € 

" 1n " 

arranged so that all the items For €* 
column are of a similar nature. ou MI 


ample, all the items in the first € 

are names of fish. jumbo" 
Scoring: The total score is the ! те 

of matrices in which all words а 

rectly arranged, d 
Paris, 1; items per part, 

time, 8 minutes; maximum score: 


12; WO 
12: 
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5. Ingenuity 

а. New Methods 

Test 26. Unusual Methods—P103A. 
Each item involves a problem which is 
ordinarily handled in ways that are fa- 
miliar to everyone; E's task is to find 
two dillerent and unusual ways of dealing 
With the problem. One restriction is that 
the ideas must be usable ones, although 
they need not be the best or as efficient 
as the methods already in use. 
Sample item: | 


jn, Best two unusual methods for accomplish- 

18 the followir 
By QITQUM = 2 

ste ш the boredom and fatigue of doing 

Steac + = : m 

mut) Y Work in a business or industry. (The usual 

em ро is to have regular breaks during which 
1 Оусез relax or take refreshments). 

lash sible answers: Have employees change their 
SKS every hour. 3 


Bee $25 : 
ie four-hour shifts instead of eight-hour 


1 Scoring: The total score consists of the 
umber І 
f mber of acceptable respouses. In order 
Or z 5 

га method to be ıcceptable it must be 


reason; : Н 
‘Sonably practical and not now in use 
to any 


Pu, ne degres: Е: 
буле " = items per рагі, 5 ЖОМЕ 
Pa nutes; maximum score, 20. 
enn Verifications—PIO2A. Сайа 
Nature зеца of a physical or a biologica 
fied, hr presented as facts to be veri- 
їйїп. ы is to find ways of аа 
Stated j^ lese facts. The phenomena а! 
nontechnical terms and satis 


жыз answers may also be phrased in 

fives, E terms, For each phenomenon 

different Is required to write down two 

ог Шушы procedures for demonstrating 
igating its validity. 

Sample item: 

at the inside 


Giy 

€ two wave А 

а vo ways of demonstrating thi " 
outside 


а lar 
Dar 8€ Bas llame is not as hot as the 


Possi P 

Ssible ; Tr 

ble answer: Hold stick in flame—outside 

ut urn sooner, 

tt they n 
hermometer in center of flame and the 


OUlsig ) F z- 
опа gare Part—temperature rises faster n se 


Scoring: The responses are weighted 
1 if the general method suggested is an 
acceptable one. An additional weight of 
1 or a total weight of 2 is given to re- 
sponses that are complete and sound. 
The total score is the sum of the weights. 

Parts, 2; items per part, 5; working 


time, зо minutes; maximum score, 40. 


b. New Applications 

Test 28. Procedure Applications— 
РІОтА. In each part Е reads a descrip- 
tion of a procedure as it is used in ac- 
complishing some particular task. A gen- 
eralized statement of the procedure then 
follows in order to make sure that all 
examinees know the procedural principle 
that is used. E must then find four other 
tasks or instances where the same pro- 


cedure would apply. 


Sample item: 

In chemistry, one of the methods for getting a 
nple of some substance is by successive 
tions. The substance is first put into 
then crystallized out. This removes 
some impurities. The process is repeated several 
times; each crystallization. removes additional 
impurities until only the pure substance is left. 
nk of this method in the general sense 


If we thi 3: 10 i 
of refining or purifying by repeating a process, 
in what other instances might this method be 


used? 
Possible answers: 
successive races от matches a 


tion purposes. 
Spelling bee—people who misspell words drop 
out until only the best speller remains. 


Rinsing clothes—each rinse takes out more 
soap until none remains. 

Scoring: Each procedure has two ma- 
jor aspects. A weight of 2 is given for 
each response in which both aspects are 
applicable; a weight of 1 is given for 
each response in which one is applicable; 
and a weight of zero is given for each 
response in which neither aspect is ap- 
plicable. The total score 1s the sum of 
the weights. | 

Parts, 5; items per part, 1; working 
e, 12.5 minutes; maximum score, 40. 


pure sar 
crystalliza 
solution, 


Athletic contests in which 
re used for elimina- 


tim 
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6. Evaluation 
a. Importance of Variables 


Test 29. Essential O perations—RPO3B. 
E is provided with a problem and five 
items of information, one of which is 
irrelevant to the solution of the problem. 
E indicates which is the irrelevant state- 
ment, but does not work the problem. 
This is a revised form of the test used 
in previous reasoning studies (3, 6), in- 
corporating certain revisions and addi- 
tions in the items that were intended to 
reduce the general-reasoning variance 
and maximize the judgmental aspects 
of the test. 


Sample item: 
Question: How many miles apart will the two 
boats be when they run out of gas? 


A. Boats X and Y each have 5 gallons of fuel, 
B. They cruise together toward the same point. 
C. Boat X uses 1 gallon of Bas going 3 miles, 
D. Boat Y uses 2 gallons of gas going 1 mile. 


E. Boat Y is twice as heavy as boat X. 

The correct answer is E, since all the 
other statements, А, В, C, and D, pro- 
vide necessary information for finding 
the answer to the question. The f 
that boat Y is twice as heavy 
does not help to answer 

Scoring: R — W/4. 

Parts, 2; items per part, 10 
time, 16 minutes; m 


act 
аз boat X 
the question, 


; working 
aximum score, эо, 
Test зо. Ranking of Variables~PJO24. 
This is an adaptation of the Awareness 
of Variables test described previously as 
test 5. Each item consists of a problem 
in which some decision has to be made 
and a number of facts that would be of 
aid in making the decision, E's task in 
this test is to rank all the facts in order 
of their relative importance in reaching 
a decision. 

Sample item: The sample item given 
for test 5 is also used here, This time 
the statements are ranked in order of 
importance with ; indicating the most 


important and 5 marking the least im- 


portant fact to consider. The — es 
signed to the statements in the за E^ 
item of test 5 are: А-4, B-3, C—2, , 
" Scoring: The score for each ча A 
determined from the sum of the pad 
ences obtained between the keyed rangs 
and the ranks assigned by the SE 
By this system, the greatest possible et 
for each item composed of seven ы 3 
ments is 24 points. (The maximum Lans 
would occur, for example, if the "a, 94 
rank order is the complete reverse of н з 
keyed order). In order to make the hig = 
scores correspond to the better mE 
the sum of the obtained differences 
each item is subtracted from 24. А 

Parts, 2; items per part, э; working 
time, то minutes; maximum score, 96- 
b. Seeing Deficiencies m 

Test 31. Seeing Dalene es 
In this test E is presented with pe 
descriptions of plans or activities t ^ 
for some reason will not lead to the y" 
sired result. E's task is to point out 1 
what way the plan is faulty. 


Sample item: 


to im- 


" - ds A 
A Browing city discovers pressing nce After 


Prove both its streets and its sewer system. rk on 

due consideration, the council decides to What is 

the street improvement program first. 

wrong with this plan? -o sewer 
Possible answer: Improving streets De arsit jo? 

System would require the street improven 

be done twice, 


: d 
Scoring: One written response 15 € 
for in each item and this must e 
spond in meaning to the keyed m ex 
No penalty is made for difference “score 
Pression of the answer. The total 5 an 
is the number of items correctly 
swered, 


vorking 
Parts, ә; wo 


А items per part, 10; né; Eê: 
ume, 20 minutes; maximum score, 


7- General Planning . 
A, Ths 
Test 52. Planning Skills-PGO14- 
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15 a test by Irving Lorge used in studying 
decision-making. The test requires E to 
devise a complete plan for meeting à 
practical problem. The problem is simi- 
lar to some of those encountered by the 
military in connection with human re- 
lations and morale. It involves the elfect 
Of the conditions at some hypothetical 
Air Force base in the Southwest upon 
the personnel assigned to the base. These 
conditions include large work load, lack 
of adequate working facilities, lack of 
recreational facilities, lack of facilities 
for dependents and relative isolation of 
the base. E's task is to recommend a 
plan of action to solve the problems of 
low morale and low operating efficiency 
of the base. 

, Scoring: The procedure used by Lorge 
In scoring the test is employed here. 
Lorge had defined, by means of a con- 
Vent analysis, eight areas in which the 
responses could be grouped. Within each 
area there are general statements listed 
that serve to classify the responses as to 
quality and determine the weights that 


Sample item: 


` 


par” 


/ 
( Sharpest Turn 
\ 


should be attached to them. These 
weights were originally obtained by using 
a large sample of responses and demon- 
strating a high agreement among raters 
in setting the weights. The total score is 
the sum of the weights. 

Parts, 1; items per part, 1; working 
time, 15 minutes; maximum score, U. 


B. DESCRIPTION OF REFERENCE TESTS 
т. Planning (AAF Analyses) 

Test 33. Planning Air Maneuvers— 
CI,OSAX5. This test assumes that Æ is 
a sky-writing pilot who must plan how 
to write two adjacent letters by flying 
the shortest possible path. The starting 
and finishing positions of the plane are 
shown, and the sharpest turn that the 
an make is indicated. With this 


plane с 
the large letters to 


information, and 
be written presented in the test, E must 


select the most direct path in terms of 
the instructions and indicate the direc- 
tion he is traveling at each of the num- 


bered points (see Fig. 11). 


16 R. 


By referring to the right-hand hall 
of Fig. 11, it can be seen that the correct 
direction for ı is A, for 2 is B, 3 is A, 
4 is B, and 5 is B. . Р 

Scoring: Since the choice of а direc- 
tion at one of the numbered points 
affects the direction choice at subsequent 
decision points in the same pattern, it 
was decided to score the test on the basis 
of total patterns. Within each pattern 

of letters, all choices have to be correct 
before credit is given. The score is the 
number of correct patterns. 

Parts, 2; items per part, 10; working 
time, 16 minutes; maximum score, 20. 

Test 34. Planning a Circuit—ClyOr. 
This is another Air Force test of plan- 
ning. Each item consists of an electrical- 
circuit diagram with many intersecting 
wires and with several sets of terminals. 
Е'$ task is to trace the circuits and deter- 
mine at which pair of terminals a bat- 
tery should be placed in order to com- 
plete the circuit through 


a meter (see 
Fig. 12). 


Sample item: 


A B с 


Fic. 12 


, Dots in the figure re that 
is, two wires are here 
there is no such d ires simply 
cross each other with no connection. Both poles 
of A are connected to the same point 


Scoring: R — W/4. 
Parts, 3; items per part, 


» : 12; working 
tune, 12 minutes; maximum 


Score, 96. 
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2. Integration 111 


Test 35. Code Analysis. Each item re 
sists of a key row ol five letters and 
numbers and ‘below it five other similar 
rows which constitute the choices. ES 
task is to select the choice row that com 
tains a given number ol symbols ol Шү 
key row. The numbers 1 to 5 are ped 
changeable with the letters A to E, i 
spectively. In other words, .\ is the wv 
as 1, В is the same as 2, Cle. rhe firs 
number, five or less (ог its eni 
letter), in the key row indicates the e 
ber of the symbols in the key row thé 
the choice row should contain. 


Sample items: 


LE wp Go N38 
A. A рй 
B.3351l3 By Ge р | 
C. HCi1ED ЖШ; 
DFIE RS eee 
E1 33H D Ecl 


Answers: Т-р, 


Scoring: R — W/4. corking 
Parts, 2; items per part, 20; b Ó 
time, 22 minutes; maximum scores, 4 


3. Verbal Comprehension - 

Test 36. G-Z Verbal Comprehensn е 
This test is Part I (Short Form) ae 
Guillord-Zimmerman Aptitude € 
It is a standard vocabulary test, prese 
ing in each item a given word ant 
alternative choices for the best sy?° 

Scoring: R — Wia. rking 

Parts, 1; items per part, 40: a 40° 
time, 12 minutes; maximum КОЛ 

Test 37. Vocabulary Test—Gl? ains 
This is an Air Force test. Jt pep 
Words of appropriate difficulty 1 lege 
of approximately two years of © 
training. 

Scoring: R — W/4, converted. 
nine value. 


ny 


to 8 


E 
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к arts, 1; items per part, 150; working 
Ime, 15 minutes. 


Г 4. General Reasoning 
bein, 4 7 diagram and observing an in- 
bun. 8 меро? of rules, Ё must deter- 
Бе} fers of three given ports is near- 
Жап = pa ate given position. In addi- 
TAR es m = of distance, the wind 
of the test ET »e considered at the start 
ports, MER Led the adequacy of the 
Hendin e current, and the initial 
iito Aa the ship must also be taken 
бако Е АП these conditions in- 

ant stance according to given rules. 

BU em W/2.‏ ر د 

Foisis ar per part, 15; working 
Test ишер тинн score, 15. 
Gort m Ч Arithmetic Reasoning— 
fest Fin de tis is an Air Force test. The 
тебе А sol the setting up of an arith- 
example ies to short problems. For 
of Esel = a plane uses a given amount 
much oo fly a given distance, how 
another ni ey would be required to fly 
correct Gs en ашап? In each item the 
five Ubri dara must be selected among 

Scori % Зэн 
nine Sed Fondo converted to sta- 
Des 
"m ^ 1; items per part, 30 working 
35 minutes. 
Test 5. арг Evaluation 
Tiis ed Togiral Reasoning-—RLRO5A. 
include pis rui y test. The pem: 
Mixed E ie mt hypothetical, pm 
Strictly eae not all of which an 
Ments are =e Гуо, three, or four sta 
amince rel aen in an item and the ex 
ог igo na ecide whether the id 
Statemen on, follows from the prece ing 
ts. 


S 
тре item: 
ll 
bombers are heavier than air. All airplanes 


" 
are hcavier than air. Thercfore, all bombers are 
airplanes. 

Answer: False. 


Scoring: R — W. 

Parts, 2; items per part, 15; working 
time, 10 minutes; maximum score, 30. 

Test 41. Inference—RDO3B. Each 
item presents a statement of fact or 
opinion and five alternate conclusions. 
; indicates which of these can be drawn 
with most justification. 'Тһе items are 
logically informal. 


Sample item: 


Most of the trees in the forest are green. 

. There are no yellow trees in the forest. 
There are some yellow trees in the forest. 
Some of the trees in the forest are green. 
. Green trees are the tallest in the forest. 
Pine trees are green. 


HDH 


Answer: С. 


Scoring: R — W/4- 

Parts, 2; items per part, 8; working 
time, 10 minutes; maximum score, 16. 

6. Originality 

Test 42. Plot Tiles (Cleverness)— 
CFO2B. Four brief story plots are pre- 
E is instructed to write as many 
ate titles as he can for each plot. 

Scoring: Two scores are derived from 
this test; а low-quality score and a clever- 
ness score. The responses are rated clever 
or nonclever by comparing them with a 
comprehensive list prepared in connec- 
tion with previous studies. The clever- 
re is the total number of re- 
dged clever. 
items per part, 15 working 
5; maximum score, U. 


sented. 
appropri 


ness SCO: 
sponses ju 
Parts, 4 
time; 12 minute: 
7. Ideational Fluency 


plot Titles (Low Quality)— 


i a 
TET above for description 


CFO2B. (See no 42 


of test.) i logget 
scoring: Total number o — qese?" 
e2! dn m Que 
9 e cid 
ә v 
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responses, i.e., responses not judged as 
clever. | 
Test 44. Consequences (Low Quality)— 
СЕОбА. (See no. 8 for description of 
test.) | 
Scoring: Total number of low-quality 
responses. 


8. Associational Fluency 


Test 45. Controlled Associations 11— 
CAFO2B. Each item consists of a com- 


mon word for which E is to list three 
synonyms, 


Sample item: 


ODD: unusual 
unique 
queer 
Scoring: Number of responses judged 
as sufficiently similar to the stimulus 
words, 
Parts, 2; items per part, 10; working 
time, 6 minutes; maximum score, бо. 


9. Adaptive Flexibility 
Test 46. Match Problems—CXO 3B. In 


this test groups of matches ar 


е laid out 
to form patterns of squares 


and triangles, 
E's task is to take away a certain number 


of matches and have a certain number 
of squares or triangles remaining, They 
must come out even with no extra 
matches left over (see Fig. 13). 


Sample item: 


TAKE AWAY 2 MATCHES 
AND LEAVE 2 SQUARES 


Fic. 19 


The correct matches to be taken aw; 
are indicated by crossing them out wi 
short heavy marks as has been done 


ay 
th 


in 
the figure above. Notice that if any other 
pair of matches is taken away an extra 


match is left over in addi 


tion to the 
two squares. 
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Some of the test items are so con- 
structed that they are insolvable if the 
subject imposes unnecessary restricuons 
upon himself, e.g., that the squares ОГ 
triangles be the same size. | 

Scoring: Number of correct solutions. 

Parts, 2; items per part, 9; working 
time, 12 minutes; maximum score, 18. 

Test 47. Sign Changes—CXO,4B. This 
test involves simple numerical operations 
in which E must substitute one arith- 
metic operation sign for another before 
performing the operation. 


Sample item: 


Р — I^ see: 
In the following problems, wherever you 
— replace it by X; + replace it by —. 


3—6=18 
64+2=— 
4—95—— 


; 4 е4 

The answer for the second item should b 
and for the third item 72. 

NS r aE Tu ses. 

Scoring: Number of correct respon " 

3 А ^. work- 

Parts, 3; items per part, 8, 16, 16; W с 

ing time, 2 minutes; maximum score, 3% 


то. Eduction of Conceptual Relations 


Test 48. Verbal Analogies I-RCRO1C- 
This is a standard verbal-analogies Vid 9 
of test. The only new idea is ап gant 
to make the relationship between the : sp 
pair of words relatively difficult to M 
and the analogy relatively easy to fu 


Sample item: 


1. electricity : energy : : green : ? 


iage 
4. color B. red C. grass D. trees E- foliag 


2 


+ athlete : scholar : ; hands : ? 
A. seclusion В. write С. study D. boo 


Answers: 1-А, 2-Е. 

Scoring: R — W/4. 

Parts, 2; items per part, 15; 30. 
ume, 12 minutes; maximum scores $ 


k E. mind 
workin’ 


rr. Judgment 


Test 49. Practical Judgment—Cl 


3016. 

е 
Ur А ires 
This is an Air Force test. It requi 
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selecti i 
е ction of the best solution to a wide 
ange of pr x : 
e of problem situations, mostly in a 
Fed setting; the items are of the 
ts commonly called "work-planning" 
ms. T ans [ 
vw The answers are selected from 
: 1рѕ of four or five alternatives. 
coring: s ' 
5i ng: 2R — W/s, converted to sta- 
une value, 
Parts. 5: 5 
‚ arts, 1; items per part, 30; workin 
ume. г да e 
» 30 minutes. 


12. Visualization 


Te " 
Cleo, 50. Mechanical Principles— 
3 is ап Air Force Classifica- 


ti 5 
i ei a all items are presented 
Район. р? Ке task consists in the ap- 
Шеш ec — principles to the 
oirett ы Е has to select the 
choices, тые among three to five 
tracting a ^ ru qe two ways of ex- 
hammer ren Bon a board with a claw 
to ا‎ and the problem is 
either, andi. ліс : of the two ways, il 
Scoring: R e force. 
stanine value V/2 + 20, converted to 
Parts ч 
reins, 1: items per part, 40; working 
› ?0 minutes, 


13. Symbol Manipulation 


Test ee. 


Sy ) > А ха 
items consis ymbol Manipulation. The 


Posed of m, 9 a main statement conr 

Severa] ,. ers and symbols followed by 

bol. O also stated in sym- 

‹ ечк of the symbols given, 

clusions j s whether each of the con- 

the Mair S true or false in relation to 
1 statement. 


S 
аре item: 


Means “ 
thane S "larger “NL” means “not 
‘striae В larger than” 

r NS" means "not 
inal ga . smaller than" 
to “NE” means "not 


equal to” 


Means < 
«р lan" 
к» 
Means « 


If x S y (x is smaller than y), then: 
хЕу 

x NLy 

xLy 

Answers: 1. False 2. True 3. False 


‚з 


uo 


Scoring: R — W. 
_ Parts, 1; items per part, 25; working 
time, 7 minutes; maximum score, 25. 


14. Numerical Facility 
Test Numerical Operations— 
CI702B. This is an Air Force Classifica- 
tion test consisting of two parts: Part I 
—deciding whether answers to simple ad- 
dition and multiplication problems are 
right or wrong; Part II—finding the an- 
swer to simple subtraction and division 

problems. There are five choices. 
Scoring: R — 3W, converted to sta- 


nine value. 
Parts, 2; items per part, 1-97, 11-78; 


working time, 1-5 minutes, II-5 minutes. 


C. Tue TEST ADMINISTRATION 


The planning battery was administered 
to entering aircrew trainees at Lackland 
Air Force Base, Texas, in January 1954- 
The sample number for which scores 
were obtained on all the tests was 364. 
The test administration was reported to 
have run smoothly and the examinees 
assumed to have worked at the 
high motivation, The 
Battery had been 


ior to the experi- 


were 
tests with fairly 
Aircrew Classification 
administered just pr 
mental battery. 


IV. TREATMENT or THE DATA 


cation tests Were scored by 


The classifi 
1 scores converted to 


the Air Force anc 
stanine values; the remaining tests were 
scored at this laboratory. 

liabilities were computed 


The test re 
on a subsample of 
values and types 9 
mates are given in T 


ооо examinees. The 
f the reliability esti- 
able 1 along with 
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TABLE r 


Means, STANDARD DEVIATIONS, AND 
RELIABILITIES OF SCORES 


Test M c Th Estè 
л. Matrix Order 14.6 4.2 ло AF 
2. Seeing Trends 9-7 45 Bi AF 
3. Sensitivity to Order 130 21 — .36 AF 
. Pertinent Questions 23.5 . 8o 
+ 1 3:5 44 o. AF 
5. Awareness of Variables 9.9 2.2 “44 AF 
6. Series . . : 
7. Effects 32-6 155 KH AE 
8. Consequences (remoteness) 9-7 4.4 33 AF 
9. Competitive Planning 19.1 6% 158 AF 
то. Symbol Grouping 113 фо у АР 
11. Contingencies 
12. Route Planning E $3 E АР 
т д! kil К 
3. Planning Skills II Ыз ас uL A 
14. Planning Elaboration I$.8 4л mn F 
15. Figure Production 39. 12 on AB 
16. Alternate Methods 2.5 
17. Match Problems П ui 45 D ИЕ 
18. Symbol Production 26.2 86 8б АР 
10. Line Drawing 188.3 13. 6 Ar 
20. Temporal Ordering 37.5 EK E АР 
21. Picture Arran; 
22: Sentence Order 18.0 Ei 
23. Outlining (Part T) 1216 E 
24. Outlining (Part IT) 19.6 57 
25. Word Matrices 6:8 К 
26, Unusual Methods 
27. Verifications j 183 АЕ 
28. Procedure Applications nate AE 
29. Essential Operations її 3 AE 
30. Ranking of Variables 59.8 3b 
31. Seeing Deficiencies 
32. Planning Skills xs ZE а AF 
33. Planning Air Maneuvers а & 3 dw 
34. Planning a Circuit 20. = ge AR 
35. Code Analysis 169 N 73 ДЕ 
f > PH | 
36. G-Z Verbal С i 
а DTE omprehension 29.6 6.2 81 OE 
38. Ship Destination бє з 1з TRA 
30. Arithmetic Reasoning se 33 KR 
40. Logical Reasoning юо 6; 10 ОЕА 
" = 5 АЕ 
41. Inference 6 
42. Plot Titles (cleverness) 64 25 67 АЕ 
аз. Plot Titles (low quality) — 12:2 87 4з AF 
44. Consequences (low quality) 13.1 3 о АЕ 
45. Controlled Associations It’ аб fa 0 АР 
ù 7 F 
46: Match Problems 7-9 
47. Sign Changes 21.6 pu +36 
48. Verbal Analogies Т 12. E 30 
49. Missed Judgment 23 ч 4 
5o. Mechanical Ргїпсї 4 ? “43 
: iples $4 ro E 
51. Symbol Manipulation 
52. Numerical Operations ES oe “8 KR 
L7 йт АРА 


* АЕ: alternate-form estimate; OE: 

man-Brown formula or Horst formal Сурап estimate; Spear. 

except for tabes of items) was applied їп АЕ a^ Contained 
or tests 8, 42, 43, and 44: KR: i 

mates; TR: Test-retést, A as а third Кибет Richardson est] 


the means and standa i 
andard deviati i 
the tests. oe 
Score distributions for most of th 
е 


variables did not appear to be signifi- 
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cantly deviant from normal and the raw 
scores were used in computing Pearson 
rs. The distributions of four of the vari- 
ables (tests 12, 21, 42, 43) were markedly 
skewed. For these variables normalized 
scores were used in computing Pearson 
rs. А 

Two scores each were derived from 
the Plot Titles and Consequences tests, 
a low-quality and a cleverness score for 
the former, and a low-quality and a re- 
moteness score for the latter. In the 
correlation of one score of a test with 
the other score of the same test, the two 
cross correlations of separately timed 
part scores were computed and averaged 
in order to arrive at a correlation value 
relatively uncontaminated by any ex- 
perimental dependence of the two scores: 
For all other correlations in which these 
tests (Plot Titles and Consequences) were 
involved, the correlation cocfficient used 
rage of those 


in the analysis was an ave : 
from two part scores, The correlation 
matrix is given in Table 2. 

Seventeen centroid factors were СХ 
tracted. The highest coefficient in each 
column of the correlation matrix p 
used as the estimate of the communally 
of the corresponding variable. After €“ 
extraction the highest residual value 1" 
each column was used as an estimate "i 
the communality remaining. The liga” 
ings of the last three centroid factors меб 
all smaller than .20 but these factors wer 
used in the later rotations in order ! 
help clarify the other factors. " 

The Zimmerman graphic, orthogo™ 


n 
* А copy of the centroid factor matrix has gi 
deposited With the American Documentation 
stitute, Order Document No, 5269 07 “ion 
Auxiliary Publications Project, Photoduplic? 25 
Service, Library of Congress, Washington m 
D.C, emitting in advance $1.25 for 35 s. 


microfilm or $1.25 for 6 x 8 in. photo on 
Make checks payable to Chief, Photodupli@ 


Service, Library of Congress. 
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method was used in rotating the axes 
(14). The criteria for rotations in the 
early stages were those of simple struc- 
ture and positive manifold. Where these 
criteria were in conflict, the rotation was 
guided by the criterion of simple struc- 
ture. After each axis had been rotated 
at least once, psychological meaningful- 
hess and agreement with previous well- 
known factor-analytic results became im- 
portant criteria. The final rotations were 
adjustive in nature in order to improve 
simple structure and positive manifold. 
The final rotated factor loadings are pre- 
sented in Table 8. 


/ 
У. INTERPRETATION OF THE FACTORS 


The factors are presented in the ap- 
proximate order of their familiarity and 
definiteness, The interpretations rest 
principally upon those tests with loadings 
of go or greater. The test numbers, 
names, and factor loadings are listed 
preceding the discussion of each factor. 
In parentheses are loadings .go or greater 
On other factors, Discussion of the refer- 
De factors is omitted except where the 
Зоната relatively less well-known. 


Factor A, Verbal Comprehension (V) 


37 Vocabular 

Жер жү 3 

36 G-Z Verbal з 
Comprehension т 


4 Сцегепсе Test 32 (30 LE, 30 Or) 
9 Ontrolled Associations 
II " 


Factor B. Numerical Facility (М) 


52 i 
ra Numerica] Operations 66 
39 gn Changes " 


Arithmetic Reasoning 43 (31 GR) 


Meg Factor C. Visualization (Vz) 

`o | CChanica] Princi 2 АХ 

19 rinciples — 44 (82 ) 

Oute Planning .32 (38 PF, 34 AX) 


Shi Factor D, General Reasoning (GR) 

35 Сой, Destination 

39 Ari * Analysis 

39 gg tic Reasoning .31 (43 N) 

33 Operations 3i (31 LE, 3! J) 
ining (Part Т) 31 


44 
35 (33 PF) 


Factor E. Logical Evaluation (LE) 
40 Logical Reasoning 46 
2g Essential Operations Зі (31 GR, .31 J) 
41 Inference Test 30 (.32 У, .50 Or) 
Factor F. Ideational Fluency (IF) 
44 Consequences (Low 


Quality) 49 
43 Plot Titles 
(Low Quality) 37 


The two tests listed for factor F were 
included in the battery primarily as 
reference tests for ideational fluency. The 
factor was previously defined (13) as the 
speed of calling up ideas in a situation 
in which there is relatively little restric- 
tion, and quality does not matter. 


Factor G. Judgment (J) 


gı Seeing Deficiencies 46 (.36 Or) 

11 Contingencies 43 (39 CF) 

27 Verifications «38 

2g Essential Operations 31 (31 GR, .31 LE) 
49 Practical Judgment 80 


14 Planning Elaboration .30 (44 E) 


The judgment factor usually appears 
whenever "judgment" tests are included 
that are composed of items presenting a 
predicament and requiring the selection 
of the best solution. In the AAF analyses 
(9) the hypothesis offered for this factor 
was "the ability to weigh solutions and 
select the wisest and best one." 

These results indicate that judgment 
is not bound to the multiple-choice test 
format. The individual himself сап pro- 
duce solutions prior to weighing the 
merits of solutions. A fluent recall of 
experiences that are relevant to the p 
lem situation would provide the in Б 
vidual with а wide range of possible 


olutions from which to choose. In ex- 
- ber of solutions toa practi- 


ining а nun а 
cal ТОЛЕП jt is not unlikely that the 
i can be made when the ex- 


best choice $ ы 
aminee considers the given solutions 


ainst the background of his oma 

aga i his 

ence placing them in a hierarchy о 
› 


solutions that he produces. 


R. М. BERGER, J. Р. GUILFORD, AND Р. К. CHRISTENSEN 
22 Я А 
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Factor H. Eduction of Conceptual inter 


Relations (CR) 


meaningful relationships, and the 
48 Verbal Analogies I 


1 
я " up?! 
" pretation of the factor was based Ш 


3 Sensitivity to Order 46 


This factor emerged in the two reason- 
ing studies (3, 6). In both of those 
analyses, Verbal Analogies [ was the 
leading test. The test was Specifically 
designed to emphasize the discovery of 


this intent, А inr 
The discovery feature is certainly j 
portant in Sensitivity to Order un ers 
this test adjustments in the given 0! 0 
of words are to be made when p 
the words is seen to be out of 0106 
Although the emergence of factor 
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aS not b ck this ability, the definition 

OL tests Sere clarified, due to the paucity 
With loading on the factor. 
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26 Unusual Methods .32 (34 9 
15 Figure Production .30 < | 

This seems to be the originality factor 
that emerged in the analysis z e 
thinking tests- In that study, P е pe 
was the leading test ОП originality, 


Consequences also had a significant load- 


ing. 
Sym 


bol Production was designed as à 
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test of symbolic thinking, along with the 
Line Drawing test (8. Both tests had 
their highest loading on originality, al- 
though in the case of the Line Drawing 
test, which had a very low communality 
the loading is only nearsignificant (26), 
Apparently the invention of symbols in- 
volves originality as much as those tests 


that call for clever, remote, and uncom- 
mon responses. 


oted 


1 
А —" р ay be ! 
interesting paradox may bo 


in that the Line Drawing and the P a 
Production tests were scored on the an 

of commonness (in the former test) p 
intelligibility (in the latter) of the ict 
bols, Seemingly, a similar psycholo8 of 
Process is demonstrated in one UP ев 
fest resulting in uncommon mu e5 
and in other tests resulting in nep * 
that аге common and readily underst 


n 
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Perhaps the conception of the factor of 
originality needs to be broadened to in- 
clude symbolizing activities. The produc- 
tion of symbols of the kinds involved in 
tests 18 and ig may be a sufficiently 
uncommon act in itself to require origi- 
nality, so that the scoring can stress com- 
mon responses and yet measure original- 
"Uh In any event, Symbol Production 
emerges in this study as a very promising 
test of originality. 
Factor J. Perceptual Foresight (PF) 


Ө аре Planning — 4o 
4 Planning a Circuit 39 (35 AX) 


E ходе Planning 38 (34 V4 34 me 
| Symbol Grouping 36 

ay Code Analysis 33 (35 GR) 

7 Match Problems II 32 (as АХ) 


to ; M factor seems to involve the ability 
| ‚ explore visually possible courses of 
= order to select the most effective 
T i getting solutions to detailed 
Reus oe Competitive Planning, 
were sel anning, and Symbol Grouping 
foresi ore or designed as measures of 
tisk С f Each of these tests presents а 
live е» шен, ostensibly, several alterna- 
Only ме ars be pursued to the goal. 
Sidered Я of the courses, however, is con- 
each н fulfill the criterion given. In 
eyond E the examinee must look well 

order t t ie step at which he is working in 
tion, Ti select the proper course of ac 
Obviou le steps or moves that are most 
Sains Ser seem to provide immediate 
are not always the best in terms 


of fu = А 
5 ture gains or in leading to а correct 
olution 


46 М К. Adaptive Flexibility (АХ) 
D Problems 57 
anning Air Maneuvers .47 


1 

"i Poe Problems II 43 (32 PF) 

12 ERE a Circuit .35 (39 PE) 

5o M, ute Planning 34 (34 V2 38 PF) 
есһапіса] Principles -32 (44 Уз) 


This has been identified as adaptive 


2 


or 


flexibility, principally on the basis of 
Match Problems, which served as a refer- 
ence test for this factor. The presence of 
three Air Force planning tests on this 
factor might suggest that this is the Air 
Force planning factor. It is much easier 
to rationalize these tests as flexibility 
tests, however, than to rationalize Match 
Problems as a planning test. This sug- 
gests that the Air Force planning factor 
was indeed adaptive flexibility. Another 
possibility is that the two factors have 
telescoped into one, but we prefer the 
interpretation stated first. 

Adaptive flexibility was defined in the 
creativity study (13) as the ability to 
change set to meet new requirements 
imposed by changing problems. In an 
1 study in collaboration with 


unpublished 
the Institute of Personality Assessment 


and Research at Berkeley there was evi- 
e of a further generality of adaptive 
in the significant loadings of 


flexibility 
two tests that required perceptual analy- 


sis and perceptual reorganization prior 
to solution. In that study it was also 
indicated that problem solving requiring 
insights may in general depend on the 
ability to shift set. 

In all the tests in the above list some 
sual restructuring in order to 
oblem seems to be a common 
is the examinee re- 
cturing but he 


denc 


type of vi 
solve a pr 
feature. Not only 
quired to do visual restru 
must also reorganize perceptually the 
same material in different ways to meet 
different requirements. In the Match 
Problems test, some of the problems are 
impossible of solution unless the ex- 
aminee breaks а self-imposed instruc- 
tional set. In Planning Air Maneuvers, 
frequently the same or similar letter 
configurations are used but the differ- 

tart and end positions 


ences in the 5 
among these configurations require the 


examinee to find new patterns. 
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Factor L. Ordering (Or) 


21 Picture Arrangement 53 
22 Sentence Order 45 
25 Word Matrices 44 


9 


о Temporal Ordering 36 

1 Seeing Deficiencies :36 (46 J) 

1 Matrix Order 33 

28 Procedure Applications .32 

41 Inference Test 130 (.32 V, зо LE) 
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This is the hypothesized ordering fac- 
tor. Tests 21, 22, 25, 20, and 1 all re- 
quire some indication of the correct ar- 
rangement of objects or events. The three 
tests constructed or adapted to measure 
the temporal aspect of ordering, Picture 
Arrangement, Sentence Order, and Tem- 
poral Ordering, all had significant load- 
ings. Of the three tests that called for 
the hierarchical arrangement of subject 
matter, Outlining I and II and Word 
Matrices, only Word Matrices 
weighted significantly. 

In the two leading tests, Picture Ar- 
Fangement and Sentence Order, a tem- 
poral arrangement of events is required 
in order to make a sensibl 
Picture Arrangement, the events are pre- 
sented pictorially; in Sentence Order, 
the events are in language form. These 
two tests were involved in Adkins’ re 
soning study (1) 


was 


€ story, In 


a- 
- In order to explain 
their significant loadings on a verbal fac- 
tor in that study, it was suggested that 
verbal comprehension would be more 
suitably named verbal relations w 
“perception and manipulation of y 
relations” is emph 


here 
erbal 
asized rather than just 
the knowledge of word meanings. The 
present analysis, with the increased num- 
ber of ordering tests, has effecte: 
ration between the verbal-co 
sion factor, defined by vocabu 
and ordering, defined by tasks 
manipulation of relations is re 

Consideration of the tests lis 
leads to the interpretation of 
as the ability to arrange objects 


da sepa- 
mprehen- 
lary tests, 
in which 
quired. 
ted above 
ordering 
or events, 


or to define an arrangement of objects 
or events, in a sequence that is ат 
ingful either in terms ol time, hierarch 
cal, or causal relationship. 


Factor M. Elaboration (E) 


7 Effects 47 (46 CF) 
14 Planning Elaboration да бз? D. 
13 Planning Skills 1I до (31 CF) 
15 Figure. Production 37 (80 О) 
26 Unusual Methods 31 (32 О) 


This is the elaboration factor hypothe 
sized to be involved in tasks uuum 
the specification of details that кен 
contribute to the development of o 
idea or to the variations of an беа, @ 

In both Planning Einhorn ТУ, 
Planning Skills II, the verbal speci p 
tion of details was called for m. е ig 
velopment of plans or activities. ае 
nificant loading of Figure Produc ion 
on this factor indicates that elabora! d 
seems to be common to both figural 4? 


zi re 1 ис 
verbal tasks. In the Figure Prod M 
duc 


tion 


lest, the examinee is required to pr ous 
à meaningful figure out of an us 
marking. The figures produced pen 
scored for the degree of mm 
that is, the more detail the exam" 
puts in, the higher his score. Effects 

The significant loadings of the 1 the 
test on this factor, a test in whic? 
examinee lists four future effects ue 
given present trend, and of the рана 
Methods test, which calls for — 
Ways of handling given probiers, rriet 
cate that idea development can be e that 
on in terms of variations of an idea 
show its range or scope. 


P F, 
Factor N. Conceptual Foresight (С ) 


4 Pertinent Questions 58 
7 Effects 46 (47 D 
16 Alternate Methods 44 
11 Contingencies 39 (43 D 
8 Consequences 

(Remoteness) 3o (34 ©) 
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T — ndings 
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hides seem to require the examinee 
бы чен) а problem situation In such 
ways that needs or consequences are 
anticipated: Tn. tests 4 and 11 the ex- 
anal gag be aware of or anticipate 
bi апп 5 oa a situation. In tests 7 and 8 
st see the more remote conse- 
еа connected with hypothetical 
] penings. 

test on factor №, Pertinent‏ اا 
он ee as an orienta-‏ 
by sica a which the examinee must,‏ 
ат enn. Mig. different questions, show‏ 
ШЕ Пы ess of the variables present in‏ 
үа а b his wasa decision-making‏ 
provided "Ph vary little information was‏ 
than an ^ m task proved to be more‏ 
ета, varones process; the examinee‏ 
tens of niga ing ma the variable in‏ 
ле needs he foresaw In the‏ ; 

Situation, 
of „ме test was devised as а test 
Сапе a ation, and, along with the 
ness. ^ де test, was scored for remote- 
aminee a two tests require the ex- 
the к, unk of various features of 
or consequen in керн of which the effects 
1S à task inar could be indicated. This 
Factors О vng foresight. i 
>y any d , and Q were not definet 
Were consid ig? greater than .30, and 

sidered residual factors. 
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anning activities and in terms 
alitative aspects of planning. It 
that planning could not be ade- 
S meque as a unitary psycho 
“anction but must be broadly con- 
Asses 2i з term. covering à number of 
SOE GP se activities and entailing a num" 
Size qn рне abilities. It was hypothe- 

lat as planning proceeds through 
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certain stages or when different aspects 
of planning are emphasized, different 
abilities are called upon, either consecu- 
tively or in combination. 


A. RELATION OF OBTAINED FACTORS TO 
INITIAL HYPOTHESES 


The hypothesis of orientation as an 
activity necessary not only to planning 
performance but to problem-solving in 
general did not correspond to any one of 
the previously found factors or to any 
new ability. Orientation remains only a 
descriptive term for those activities that 
involve the seeing of an order or trend 
in a mass of information and that show 
an awareness of the pertinent variables 
operating in a situation. Tests of the 
two subhypotheses of orientation are 
loaded variously on factors of ordering, 
originality, eduction of conceptual vela- 
tions, and conceptual foresight. 

The futuristic character of planning 
was hypothesized to call 
upon à predictive ability. Tests of predic- 
tion were constructed in terms of two 
s of prediction: extrapolation 
For each of these varia- 
al and conceptual 
analysis. А 


performance 


variation: 
and foresight. 
both perceptu 
tests were included in the sis. 
separation occurred in terms of this dis- 
tinction rather than between extrapola- 
tion and foresight. Competitive Plan- 
ning, Symbol Grouping, and Route 
Planning, designed as perceptual bui 
involving foresight, emerged on per 
resight. 
irap of a perceptual foresight 


which is defined in part by maze 
dance with С. D. Porteus 


e maze tests he used were 

foresight (12). Porteus 
ore than опе level 
je, that there 
pified by the 


in solving the mazes, 


tions, 


factor, 
tests, is jn accor 
pelief that th 
tasks involving ^? 
implied that there 15 m 
ight ОГ planning, 
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and a more abstract or high-level kind, 
as called for in creative work, and that 
it is necessary to be capable of the lower- 
level foresight before one can exhibit 
the higher-level type. The results of this 
study would seem to confirm the ex- 
istence of more than one kind of fore- 
sight. Perceptual foresight and concep- 
tual foresight, however, were found as 
distinct factors, indicating that one type 
of foresight is not necessarily basic to the 
other. 

The three conceptual tests constructed 
to measure the predictive ability helped 
to define the conceptual-foresight factor. 
Foresight in this study has been viewed 
аз ап awareness of possible future events 
that have a relation to a present situation. 
This definition of foresight appears to 
involve the flexibility of ideas, as indi- 
cated by the fact that the examinee has 
to conceptualize different needs that 
might be connected with the given situ- 
ation in order to answer the problem 
in the foresight tests, 

The results do not seem to support 
the subhypothesis of extrapolation as 
one type of task involving a predictive 
ability. The Series test, intended as а 
perceptual task requiring extrapolation, 
was not significantly weighted on any of 
the factors in this analysis. As indicated 
in the interpretation of the conceptual- 
foresight factor, the Effects test and Con- 
sequences could be characterized more 
as tests of foresight than of extrapolation 
because of the additional information re- 
quired in order to give suitable responses. 

Due to the multiple-response character- 
istic of most of the conceptual-foresight 
tests, it is not clear to what extent re- 
sponse fluency enters into the factor. It 
remains for future study to clarify this 
aspect of the factor. 

The identification of a factor of elab- 
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oration was based upon the significant 
loadings of the tests connected with the 
subhypothesis of specification. It is an 
interesting finding that elaboration is 
not confined to verbal tests, as is indi- 
cated by the presence of a figural test on 
this factor. Apparently the ability tO 
elaborate in a figural task is related s 
the ability to specify details in a verba 
task. Had there been more alternate 
tests. requiring perceptual м 
however, a factor for such tests ш>» 
have been split off from опе for con- 
ceptual elaboration. е 
ae significant loading of the n 
test on this factor indicated that 1 
factor should be described not only г 
terms of specifying details in order a 
develop an idea but also in terms g adi 
production of variations of an idea wil 
serve to show the scope of the idea. E 
Some comment should be made (n 
the subhypothesis of symbolization. a 
elaboration factor apparently does v 
encompass the symbolic represen маар З 
ideas; neither does the symbolization foli 
hypothesis define а separate | gine 
tion factor. The expected high pim 
tion between the two tests, Symbol f a 
duction and Line Drawing, did not m 
terialize. The variances of these two E 1€ 
Were accounted for principally by 
originality factor, „ хае 
The confirmation of an qu ad- 
tor is indicated by the significant ye 
ings for tests that contained en 
quiring the ordering of bits of qnt 
tion in temporal, causal, and 108 
Sequences. think 
The tests designed to measure ning 
ing ingenuity in relation to pla” ас 
tasks did not serve to determine ance 
tor of ingenuity. Much of the et n 
On these tests, particularly that O rhe 
usual Methods, was accounted for PY 
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originality factor. There was a question 
whether originality, defined in terms of 
BAGG entionality of thinking in the 
Grentivity study, would be important in 
planning tasks where new procedures, 
s applications of old procedures to new 
генг и һе devised. Admittedly 
fac желе — ol ingenuity was not satis- 
нун. : sted in some respects. There 
malade i tests representing this hy- 
if the е н it might have been better 
Ий des, аш features in these tests 
m reduced. 
Md: ше three tests designed to 
йїп с evaluation aspects of plan- 
rged on the judgment factor. 


B. RELATION or RESULTS TO FACTORS IN 
PREVIOUS STUDIES 
е — were included in the 
factors 2 oattery Гог fourteen reference 
lors, ма i Аа of the reference fac- 
MEME CHE fluency, symbol ma- 
emerge On, and integration 11, did not 
Tie win some form in the analysis. 
the AAF ms planning factor found in 
bility не теге and ої adaptive flexi- 
emerged test in the creativity study, 
(adaptive together on а single factor 
to aim; flexibility). This result helps 
тере 10 ѕоте degree the queen 
Senteq s e essential ability терге- 
also ie the AAF planning factor. It 
tive adens the conception of adap- 
£ flexibility, 

and scop sja with regard to the na І 

bes of certain other reference fac- 


lors 

S: ы : 9 à 

in y. Judgment was found to be weighted 
as well as 


were also 


infoy 
tures 


sg wei iua pas Lm tests 
Actor н. a multiple-choice format. The 
Ply Bets ideational fluency seems to ap- 
üre Pioda to tasks in which рер 
Conditio uced under fairly unrestrictec 

пз and involve responses that 


arise from the more superficial aspects 
of situations. 

A new test, Symbol Production, seems 
to provide a good measure of the origi- 
nality factor, previously isolated in con- 
nection with the creative-thinking bat- 
tery, and indicates a possible symbolic 
aspect of originality. 

Elaboration, ordering, perceptual fore- 
sight, and conceptual foresight are re- 
garded as new factors, but we should con- 
sider their possible relations to previously 
known factors. The verification of these 
factors, of course, will depend on future 
research. Elaboration may represent a 
new and more significant dimension of 
ideational fluency. The factor interpreted 
as perceptual foresight might alterna- 
tively be identified as the integration III 
factor found in the AAF analyses, since 
several of the tests that helped to define 
or were loaded significantly 
on factor J of this study. However, the 
leading test on integration III, Plan- 
ning Air Maneuvers, was included in this 
study but did not have a significant load- 
ing on factor J. In view of this and of 


the fact that three tests designed to meas- 


ure foresight had significant loadings on 
factor J, it was felt that this factor would 


be better characterized as perceptual fore- 


sight. 
Conceptual fore 


blance to the spo? 
tor identified in 
study, but the idea 
conceptual foresight 
or goal-directed. 


C. RELATION OF 

GENERAL PLA 
e test of Planning Skills (test 
g2) was included in the analysis as а 
general planning task. Given a detailed 
account of a practical problem situation 


the latter fact 


sight bears some resem- 
ntaneous-flexibility fac- 
the creative-abilities 
flexibility involved in 
is more purposive, 


RrsULTS TO THE 
NNING TEST 


The Lorg 


R. М. BERGER, J. Р. GUILFORD, AND Р. К. CHRISTE 
30 d 


the examinee had to define and structure 
the problem for himself, find the critical 
areas, perhaps evaluate the alternate 
methods he thought of, and write down 
all the steps necessary for carrying out 
the procedures adopted. Because of the 
complexity of the task a number of abili- 
ties were considered to be involved. 

The Planning Skills test was treated 
as any other variable in the analysis and 
no special attention was given it in the 
rotations of the axes. Results from the 
preliminary testing with two problems 
(only one problem situation was used in 
the final battery) had yielded an alter- 
nate-forms estimate of about -40. If this is 
à reasonable estimate, practically all of 
the true variance was accounted for in 
terms of common factors by this study. 
Since the factor loadings are all very low, 
however, our conclusions as to factor 


content are merely suggestive; certainly 
not decisive. 

As expected, the common-factor vari- 
ance represents a number of factors, The 
leading factors and the proportions of 
common-factor variance that each ac- 
counts for are as follows (these 


propor- 
tions add up to 1.00): 
Factor M. Elaboration 22 
Factor І, Originality 15 
Factor С. Judgment 44 
Factor F, Ideational Fluency лә 
Factor Р, (Residual) a8 
(Others) 49 


The Planning Skills test 
siderable detail in the exa 
of a plan, hence the elabor 
Responses were scored to s 


called for con- 
minee's report 
ation variance. 
ome extent for 


quality, therefore the production. of 
clever and uncommon ideas would be 
likely 


to receive weight and thus intro- 
duce variance in originality. The contri- 
bution of the factor judgment can be ra- 
tionalized by supposing that the exami- 
nee had to select critically the significant 


SSEN 


features of the problem, to evaluate x 
hypotheses and ideas that came up, me 
to maintain continual check on the in 
ternal consistency of his plan. The weight 
in ideational fluency probably reflects v 
fact that quantity of ideas was — 
in the scoring. As indicated before, the : 
small contributions may not be very n 
curately estimated. It should also be «d 
membered that the Lorge test represen 
only one kind of planning. 


VII. SUMMARY 

The purpose of this study was to нод 
late and define the abilities penne 
planning. A battery of 52 tests MUN 
six hypotheses and a number of gerer " 
factors was administered to 364 USAF а | 
crew trainees. Included in the battery e 
one test that represented a aique ai 
ning activity. The scores were in nt 
14 
ad 


related and 17 factors were extr 
Orthogonal rotations resulted. E h 
identifiable factors, ten of which pre 
been found previously: verbal ae 
hension, numerical facility, visualizat : 
evaluatto» 


Seneral reasoning, logical mz 


ideational fluency, eduction of = and 
tual relations, judgment, origin 
adaptive flexibility. Four new ` tions 
were defined as ordering, ШШ fort 
perceptual foresight, and concept the 
sight. The greatest proportions н 
common-factor v; ‘by the 
planning test were contributed | 
factors of elaboration, originality» 
ment, and ideational fluency. gatio? 
It may be said that this an 
Supports the hypothesis that 1n p1 ary 
we should find a large number of Qo idi 
abilities involved. Of the factors am 
this investigation, [our are new ау ar 
to be unique to planning tests. 1 Е Й 
the abilities of ordering, elaborati??? p. 


ceptual foresight, and conceptu? 
sight. 
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Infant Development Under Environmental Handicap’ 
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Brooklyn College and the 


M (10, 11) and Spitz (12, 13, 14 
жы boh d sai a atm that if cer- 
childhood га us ipn occur in early 
Mid апын v ias gae are drastic 
largely ча » hese views. have arisen 
Stitutions TL serian of infants in in- 
consisted * i supporting evidence has 
Subjects оп me of scores of institutional 
general imm ant tests and in part upon 

pressions of the emotional 


States 
an of the children. 
Us report i А 
havioy port is concerned with be- 


Whose Moment in an institution 
identical ae — is in some respects 
different Е. 1, and in some respects quite 
Studies, rom, that described in other 
The da 
MC dunt obtained in a foundling 
Of inadeg u i^ Lebanon, which, because 
Provide fedes financial support, is able to 
саге, We € more than essential physical 
Mental Will report upon the develop- 
dren į Status of two age-eroups ol chil- 
П this instituti b. нейн 
etw ПЧ 12 mo ion: those between ? 
icri; 4% and "Ps of age, and those 
ang Ё the envi years of age. After de- 
the Presenting ишеп conditions 
i he data we will discuss 


elati 
onsh} 
ey Nship of this st ү 
udy to previous 


his ү 
Univer ; esearch yy, 
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oiler Вт тауса p, he cost of the investiga- 
Е Betray ете" Fu y a grant from the Rocke- 
ae Prec} t The w ind to the American University 
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studies, and to theories of child develop- 


ment. 
THE CRECHE 


The institution in which the study was 
conducted will be called the Creche, al- 
though this is not the formal name of the 
home. The Creche is a home for infants 
and young children operated by a re- 

f nuns). All children in 


ligious order (0 
the Creche are received shortly after 


birth. They arrive via two routes. The 
majority come from a maternity hospital 
operated by the religious order referred 
to previously. An unmarried woman 
being attended by this hopsital may ar- 
nave her infant taken to the 
she relinquishes 


cle 
visit it th 1 1 
pulation consists of infants left 

doorstep of the institution. 
‘uç is known definitely concerning 


but it is likely that the 


iv 7 15 
their parents | x i 
m as of these infants, too, are il 
та 
legitimate. 
à ears old but it has a 


ne, being fireproof, 
her pieces 


is extremely limited, Understaffing is the direct 
cause of whatever deficiencies may characterize 
the child-care practices to be described later, 

Naturally the number of children in the insti- 
tution varies from time to time with the adyent 
of new arrivals, and departures due to deaths, or 
to transfer to other institutions to which the 
children are sent at about six years of age. The 
size of the staff, too, is subject to some variations. 
However, estimates made at two periods sepa- 
rated by five months agree in showing that for 
each person directly concerned with the care of 
the children—i.e., those who feed the children, 
change diapers, bathe and clothe them, change 
their beds, nurse them when they are ill, super- 
vise their play, and teach them—there are 10 
children. This ratio of 1 to 10 includes those 
on night duty as well as on day duty. It does not, 
however, include personnel who work in the 
kitchen, laundry, and mending room, nor those 
who do the cleaning. It does not include the four 
nuns who constitute the administrative staff and 
who frequently assist in direct care. Clearly this 
is an extremely limited staff. The essential func- 
tions can be accomplished only by means of 
hurried procedures and long hours of work. 

From birth to one year there is no assignment 
of individual children to particular attendants. 
Rather, a room of children is assigned jointly 
to several caretakers and observation showed no 
consistent relationships between attendants and 
children. At later ages, each group of children 
is assigned most of the day to a Supervisor and 
„an assistant, 


During the first two months of life the infant 
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Я dv 1 
is taken out of his crib only for his am m 
and change of clothes. He is given p EE 
while lying on his back in his crib, m ЫНДЕ 
narily по one has time to hold ue А sropped 
is placed in his mouth and the bott pes Lm 
up by a small pillow. Bathing and d A ta tal 
done with a maximum of dispatch ап 

mum of mothering. eastern 

In conformity with a widespread Rear rn 
practice, the infant is swaddled ee ШЕЙ! 
Figure 1 illustrates the type of a legs ef 
Ihe baby has his arms as well as bi scope © 
closed in tight wappings, and hence aiig the 
his movements is greatly restricted. сыкса ex- 
early weeks the infant is bound ere fixed 
cept when being bathed and dressed. Jom from 
schedule is followed in regard to en frecd at 
swaddling, but in gencral the hands a is endet 
about two months of аде, and swaddlinE иё 
at about four months. Swaddling !5 P months 
for a longer period during the ee because 
than during the remainder of the 5 keep him 
the wrappings of the child serve to 
warm. : 

As shown in Fig. 2, cach crib ha $ 
around the sides. This is present to P 
child from drafts, but as a bye wh 
can see only the ceiling and the acu 
casionally come near him. ү 

The adults seldom approach Lape do 
feeding times, When they feed him Vhen two 
usually speak to him or caress И any 
or three persons are feeding twenty d 
of them crying, there is no tendency 

At about four months of age Hel i place 
moved to a room for older infants. m months 
in a larger crib, but for several fur it has bee™ 
his care remains much the same as ! 


dais 


а covering 
otect the 
the child 


‚ except ® 


fants, " 
to dally. 
child is 7€ 


ی س 
Ec ———— —‏ 
анинин „СС‏ 


IN Ч 
NFANT DEVELOPMENT UNDER 


A typical 
Agee scene in the room i oe 
SI pe ob кш 
child d e n a place RE mu 
i i т i 
oe crib aul aie о remains in 
out, At deer faces some as faling 
his waking diat n is usually placed di 5 
m a play ке d one or two other гє 
Fig. 3 Sonictiin : x: situation is illustrated. in 
is usually air, as shown in Fi is 
чы older 58, only for short релш ея 
Phy pens. He is takes his daytime naps in the 
га tightly oo anaes to his crib at night 
a m UNS Тосайлой нь The child graduates from 
siig uy егез? лег room at one year of age OF 
older child ter, Some description of th Ете 
‘ Until abou ren will be given on later e 
ood Md ie UE months of age Ehe infants 
E le feedings ed supplemented by vitamins. 
d is feedings ng this time are on a schedule 
а hours, dier | day at daytime intervals of 
gradually кше аг months bottle feeding is 
ve months. in frequency. It ceases at about 
m introducti 
xi months, bras of cooked cereals begins at 
Pie Sauce: and fruit juices, crushed bananas, 
‚ and vegetables are begun at five 


Month 

is. D 

a е i 

pj ant IE upon thc preferences of an 
„emen d is sometimes given these Sup 


а 
Sittin, агу food 

. 1 s 1 
sinning a айк hag in arms, sometimes while 
are &teisht n ometimes lyin, у 5 
Occasionally months, eggs анд ae x d 
given. Feeding times ipe а 


lve ti 
н t 
limes are per day at fi 
r - a о! 
ur months and to four 


һе 
gun | Чау д 
ie t one vear. А A 
гоч Ie Toilet training is 
area 2 months. 
eighed à Е 
ghed at weekly intervals, Seri- 


vildren 


Ес. 8 


ENVIRONMENTAL HANDICAP 
3 


emn 


Fic. 4 


ous efforts are made to giv i i 

infants who are not el pp € 
staff limitations make it difficult for i се ue 
to spend much time with any one oe 
average weight during the first six months, b 
on records of the infants which we tested, i Te 
»reciably below what is ordinarily бна d 
sirable (see Table 1). Comparable data are 7 
available for other Lebanese children. No A 


are available on children beyond six months of 


age at the Creche. 
t one year to about three years the 


uch of the day in play groups 


children spend ™ 
of about twenty children With А supervisor an 
nt is limited to à few balls, 


wagons, and swings. 


age muc 
tables. Т 
way with slates, 


within the 

jects and pictur iting, reading, 
js begun. Instruction is given in both Arabic and 
he super 


French. 
Diet and edical € are under t 
vision of a nysician who devotes, gratis, about 
one hour per day to th Creche. whose popula- 
tion is 4 out 140 children During the winter 
months colds are © mmon, and eumonia OC 
casionally occurs: Th usual childhood illnesses 
оссит. n à contagious disease enters the 
Creche it ikely tO become widespread since 
there are facilities for isolati! of infectious 
cases. not have айса te statistics on 
mortality. our im ression tit 1S high in 
{һе fi nonths of life, but P t paries 
nigh A Mortality § pecially ig) 
who 2! the door- 
among ni from malnu- 
step, ™ of whom a" f 
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TABLE 1 
AVERAGE WEIGHTS OF CRECHE INFANTS uu 
Boys 
pus < о. 
rs Birth I mo.* 2 mo. 3 mo 4 mo. 5 mo 6m 
i бе 2 5555 5984 
Average weight in grams 2926 3233 3746 4365 4926 5555 99 
Number of cases 28 28 28 27 23 eee 
Girls = 
Г 10. 
Birth 1 mo. 2 mo. 3 mo. 4 mo. gmo — oe 
z pu z463 
Average weight in grams 2727 298$ 3353 3858 4430 4910 54 3 
Number of cases 13 12 13 13 11 D з аме M 


М ; Ре red. А 
? In computing this average, for each child the record of weight taken nearest age 1.0 month was employed 


procedure was used at other ages. 


trition, exposure, or disease upon admission. In 
evaluating institutional mortality it should be 
noted that in some areas of Lebanon the crude 
death rate in the first year among children in 
homes is as high as 375 per 1000 (6). 


THE Comparison Group 


For comparison with behavioral rec- 
ords of the Creche infants, data were ob- 
tained from children brought to the Well 
Baby Clinic of the American University 
of Beirut Hospital. All well babies of ap- 
propriate age who were brought to the 
clinic on certain days were tested. They 
were from among the poorer, but not the 
poorest, segments of the Beirut popula- 
tion. 

All children tested were living at home and 
were brought to the clinic by their mothers. The 
majority were being breast fed. We did not ob- 
tain detailed data on swaddling, but typically the 
younger babies were brought in swaddled and 
the older ones unswaddled. It is our impression 
that swaddling customs among the poorer half 
of the Beirut population approximate those of 
the Creche. This conclusion is supported by a 
study by Wakim (17). Other comparison data 


were provided by American norms and certain 
Lebanese norms to be described later. 


THE TESTING PROGRAM 


For the subjects under one year of age 
the Cattell infant scale was employed 
(2). This scale was selected because 
among available tests it seemed to offer 


similar 


the most objective procedures for aame 
istration and scoring. It provides fiv 

items for each month from twọ n 
months of age, with one or two alternat 

items at each age level. 

The procedures describ hey 
manual were carefully followed. р 
call for testing each infant at а ы e 
which he passes all tests, at a oe 
fails all tests, and at all interme“ 
levels. 

Several items on the test 
plicable to the Creche group becau 
require the examiner to obtain 1n 
tion from the mother or other саг 
Among such items are babbles, 
pates feeding, inspects finger? - che 
"dada," etc. Attendants at the Grs Jes 
could not supply the information pth 
quired by these items. For this ws не 
“alternate” items provided by Catt 
based on direct observation Wen 


ed in the test 


were not ар” 
se they 
forma 
егакег: 
antici- 

says 


e 
> n th 
larly substituted for these items. i ag 
case of the comparison infants, 2 alter 


appropriate items, including all ut 
nates, were administered; but in чык 
ing developmental scores for comp? por 
purposes identical items were Soup 
the Creche and the comparison 8" us 

At the 414-to-6-year level the ien test 
were the Goodenough draw-a-ma 
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the Knox cube test, and the Porteus maze 
test. These were chosen because it was 
Judged that they might be but little af- 
fected by the environmental handicaps of 
the Creche children. They have the fur- 
ther advantage of requiring a minimum 
of verbal instructions. 

In giving and scoring the draw-a-man 
test, Goodenough procedures (5) were fol- 
lowed. For the other two tests the proce- 
dures and norms employed were those 
om in the Grace Arthur Scale of Per- 
ormance Tests, Revised Form II (1). 


NUMBER OF SUBJECTS 
pectic all subjects who fell into our 
"m ег in upon two series of testing 
children 6 опу exceptions consisted of 
‘Reisner who were ill or who had just 
{ешь к ne serious illness. The infant 
the EE gwen to 49 Creche infants and 
of ka comparison cases. Since rather few 
months Creche infants were above six 
peri 15 of age at the time of our first 
ing er testing, during our second test- 
Six on. we tested all infants who were 

lis Duns st age and over even though 
Teason ih retesting in 13 cases. For this 
49 Preis number of test scores for the 
In hee infants is 62. А 
Enough ae Yo to-6-year group, ee 
and the ^ — given to 30 subjec А 
аге test a cube test and the Porteus 
Опе wa, EI cach given to 25 subjects- 
etested. 


For the ; RESULTS 
те Ses infants, Table 2 indicates by 
he че score earned on each test. 
$ S, the ion Scores are shown by O-Sym 
Cores ay mparison scores by X-symbols. 
En Poin © grouped by step intervals of 
+ аре Thus, examining the figure by 
ands tle P the top of column опе, one 
8e One um 2.0 and 2.9 months of 
parison infant had a develop- 


mental quotient between 140 and 149, 
two comparison infants had quotients be- 
tween 190 and 199, etc. 

Examination of Table 2 shows that at 
the two-months age level there is little if 
any difference between the two groups. 
The mean of the Creche group is 97, that 
of the comparison group 107. These 
means, each based on only 8 cases, are not 
significantly different from each other or 
from the American norms. However, at 
all ages beyond 3.0 months the Creche 
infants score definitely lower than either 
the comparison or the normative groups, 
hose records are indistinguishable. 

If all scores from 2 to 12 months are 
averaged, the Creche mean is 68, the com- 
parison mean 102. For the 3-to-12-month 
eriod the mean of the Creche scores is 
63, (SD 13), that of the comparison group 
101 (SD 15), а difference of 38 points. 
This is a very large and highly significant 
difference (Р < .001). In this age range 
all of the comparison infants tested above 
the mean of the Creche subjects and all 
of the Creche subjects were below the 
mean of the comparison group. No 
Creche baby between 3 and 12 months 
had a DQ above 95: 

Before discussing 


W: 


the results of the in- 


fant tests We turn now to the tests given 

a 

t Creche children between 4-5 and 6 

: of age We note first that there are 
« of age 

үк e that the subjects tested 


reasons to believ 
at 4-5 t о yea agi performed, as 
4. о 6.0 ears of age е 
infants. at the same level as did the chil- 
› 


е test results have just been 
e, e procedures of admis- 
Creche have not changed in 
years the two groups сч з 
enter the 
there are по 


ces between admission 


уе influen ` in бї 
selecti cuis, THE only qualification 
and six Y€ regards jnfant mortality, 


this statement 
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TABLE 2 
INDIVIDUAL INFANT SCORES BY AGE* А س‎ 
Age in Months — 
1 II 
Scores : : - : ; = " 
2 3 a 
| 
140-149 x 
130-139 it а x a}. 
120-129 u - Е 
тіо-119 | OO х ۳ XX x) ci x - 
100-109 | OOXX | XXXX | X XX x. А 
90-99 00 O XX О: ^ 
80-89 оох ox XXXXX m 9 оо ° 
55 0 000000 ооооо о х |o јо [9 | Bo 
д 5 00 000 
55-40 : o OO | 000 |00 LN ИНЕ 
® Creche infant scores are indicated by O; comparison infants by X. | 
: aze tests bU 
whose selective action so far as psycho- Knox cube or Porteus maze 


logical tests are concerned is unknown, 
here as elsewhere. According to the super- 
visory staff there have been no changes 
in child care within the past six years. 
The results of the performance tests 
are shown in Table 3. It will be noted 
that the data there reported agree re- 
markably well in showing that on these 
tests the development of the Creche chil- 
dren is only about 10 per cent below the 
norms of American home-reared children. 
In a separate report (3) it has been shown 
that on the Goodenough test Lebanese 


children at the five-year level make scores 


equivalent to the American norms. No 
Lebanese norms are available for the 


TABLE 5 
RESULTS or PERFORMANCE TEsts 


Various DQ" Scores 
Test 

N Range Median Mean SD 
Goodenough 39 58-136 93 2 
Porteus maze® 25 69-150 89 5 20 
Knox cube? 25 тоо 


8 Four children earned fewer than 


minimum score for which 


there is no reason to believe e E 
would be higher than the ри? "m 
standards. In other words, there "udi 
dence that the environment of the cate 
produces only a slight retardation in 
four- and five-year-olds on these a with 

In summary, the data show m dil 
respect to behavioral developmen’ ag the 
dren in the Creche are normal pee ы 
second month of age, arc greatly 1 almost 
from 3 to 12 months of age, ant it be 
normal on certain performance t€ 
tween 4.5 and 6 years of аре. 


SION 
INTERPRETATIVE DISCUSS! 


s pu 
To a reader acquainted with me cot 
merous and often divergent nae 
cerning the effects of early ater sul” 
the results just reported p era the а” 
face, only serve to confuse ee now 
ready unclear picture. We oper data 
ever, that we can show that t? here? 
and others can be fitted into a СО 
view. 


ants 
Early Normality of Creche Inf 


rants DA 

The fact that the Creche uA the 

DQ's of approximately 100 du! quri”? 

second month, and presumably ur 
the first month also, should not 
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vis} 
Ox deter bes been shown that any 
Ка ae ima will affect infant be- 
Sünde Tha a dt doing the first two 
ШЕШ com — under experi- 
Presa ж у Dennis and Dennis 
ane rmal progress during this 

period. The infants tes 55 
iul руни : s tested by Spitz (12) 
pides the — M quotient of 
normality of eigen month. The super- 
lí бе ше is score was probably due 
than to аа, of test norms rather 

If it is oo influence. 

Nas little e that restricted stimulation 
havioral q a no effect upon early be- 
atleast oe this can be due to 
"on would b erent causes. One explana- 
erhaps — "i in terms of maturation. 
Apart from vth of the nervous system, 
“sponsible "e stimulation, is alone 
growth йон ог postnatal behavioral 
Second ex oh the first two months. А 
that bien planation lies in the possibility 
that for P experience is essential, but 
infant as ле tests presented to him the 
en when swaddled hand and 


Oot 

.-and |у 

“lent sti ying on his back obtains suffi 
mulation 


T the С 

Brera: ЛЕ Са n 
lation is т; tests the second inter- 
leve] Sts which > together unreasonable. Of the 
his y Our are hus employed at the two-months 
The апа бун to the infant while lying ОП 
ing с е “inspe responses required are visual. 
Person," folk environment," "follows mov 
infants , 9WS nn moving ring vertically." 
Positig Spend RE horizontally.” Since the 
in a well ad 24 hours per day in a supine 
Ccurs near ighted room, and some move- 
nity to them, there is considerable 
practice visual pursuit move 


Чо : 
ite alone , OWe' x 
temg Лез буе the possibility that matura- 

s icient for the devclopment of the 


п 
Ot ruled out 
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of age are relatively una 
are not propped up i 
placed in chairs before t 
occasion for placin 


Retardation Between Three and 
Twelve Months of Age 

| Beyond the two-months level the ma- 
jority of items on the Cattell scale re 
that the infant be tested in a sitting posi- 
tion while being held on the lap of an 
adult. Sitting is a position to which the 
Creche infants under about ten months 


ting position may come 


duction of semiso. 
noted that some O 
these while lying 
consequ 


held upright t 
a poor record on t 


volves holding the 


rence of i 
he infant 
he test item W 
head erect and steady. 


of the head, plus gen- 


This unsteadin 


eral unfamili 


count in par 
certain purely 


“regards cube, 


ball,” and “regards P 


Many of the ! 
not only sitting 
skills directed 


arity W 


ess 


t for the low 


are “picks UP 


gr 


, 
cup, 


asps 


pellet, 
» “takes 


15 аге not 


visual it 


“regards spoon,’ 
ellet.” 

ng items involve 
manual 


remainir 


by vision. 2 


spoon, 


" “grasps string, 
cubes,’ 


two 


require 


but in addition 
mong the items 


isuo- manua 


quire 


ccustomed. They 
n their beds or 
hat age. The first 
g the infants in a sit- 


with the intro- 


lid foods, but we have 
f the infants are given 

down. Perhaps as а 
nexperience in being 
s as а group make 
hich in- 


ith sitting, may ac 


scores earned on 
ems. These are 


Ar 


' “follows 


picks up cube,” 


” 


“lifts 


4 "exploits 


» etc, Between 


practice 


a sitting posi- 
] coordina- 


ired or encouraged 


CI & 


tween. `` 
S 
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the items and the environmental restric- 
tions experienced by the children account 
for the low scores made by the Creche 
subjects. 

We examined the records made by the 
Creche children aged з months and above 
on each item, expecting that one or two 
items might be found in regard to which 
their performance is normal. We were 
able to find none. But we were also un- 
able to find an item in this age range 
on which the subjects were judged to 


receive a normal amount of relevant ex- 
perience. 


The second item is 
-bell is used instead of 
as an age placement 
, five months. Of 36 
© and 104 
age only one turned to the n pe р ki 
turned to the bell, е 
Now all of the children t 


the second 
etween 4 


€ have noted 
iding services 


From the preceding discussion it will 
be obvious that we tend to attribute the 
retardation of Creche subjects between 
and 12 months of age to a lack of eain 


ing opportunities relative to the Cattell 
test items. 


Relationship of the 3-to-r2-Month 
Retardation to the Findings of 
Other Studies 

There seems to be a superficial, if P 
à basic, disagreement between the -— 
here reported, and those of other stut a 
particularly those of Dennis and er i 
(4) and those of Spitz (12-16). We wis ces 
comment on the apparent divergen Р. 
and to indicate how they can be reco 
ciled. 

In a study of a pair of twins n - exe 
Del and Rey who were reared anier 
perimentally controlled conditions pad 
thirteen months of age, Dennis and e 
nis found that, while the subjeds и 
retarded beyond the range of orc de 0 
subjects in regard to the a 
a few responses, the subjects pone 
ment in general equalled that ph arda- 
reared infants, The few specific = 
tions occurred on items in respec 
which the infants could not ach 
self-directed practice, namely, € sup” 
directed reaching, sitting without 


t 
ith the feei 
p will 
How 
ind 


che 


amed 


port, and supporting self w 
These retardations scem consonan 
the behavior of the Creche sj E 
ever, the prevailing normality of er 
Rey seems at variance with the 

findings, ences yi 


. " "tain differ 
lo begin with, certain ition 


tween the environmental cond uld 

the subjects in the two studies ШЕТ, 
noted. For опе thing, the adu ^. dif 
ratios in the two studies were bes y ert 
ferent. In the Del-Rey study ther. a 
two subjects and two experimen adult 
one-to-one ratio. In the Creche, he 7. ger- 
child ratio is one to ten, a greatly г ре 
ent situation, In the Dey-Rey 9! 74 10 


А em in re 
environmental restrictions in 16 


2 
атс, 
1 agin 


ind 
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and practicing responses continue 
throughout the first year. Certain specific 
contrasts may be mentioned: Del and 
Rey were kept in larger and deeper cribs, 
were less restrained by clothing and con- 
sequently probably had more opportu- 
nities for motor experimentation than 
i the Creche infants. Further, Del and 
ul be nave received more handling 
did eee exposure to stimuli than 
cai bê гесһе infants. However, there 
Del an E doubt that in several respects 
tion oe ii suffered as much а restric 
mms experience as did the Creche in- 
Rep — was not directed to Del and 
afica a s id adults smile in their pres- 
Ns to ga ча were six months of age. 
ou es provided until the twelfth 
Position ley were not placed in a sitting 
Ed until they were over eight 
nths of age. 

cc : our belief that the difference 
and ie * le normality of Del and Rey 
is due a of the Creche infants 
Cheer es use of different indices of 
rea] “зин development rather than to 
ey Mu e in behavior. In the Del- 
velopme y no general scale of infant de- 
jority a ew was administered. The ei 
for De] м Я developmental data reporte 
cach dii. Rey consisted of noting qum 
Sponses Chen of ome im = 
Servers buius асн beg ste 
Tought hand edens dy inm d bed- 
Clothes E d to mouth, first graspe Е 
Augheq irst vocalized to person, 28% 
rence of etc. The initial date of ar 
termined ар responses cannot iae 
аге longity y testing. The Del-Rey if 
idinal and the Del-Rey recores 


Wen 
e 
Те found to be normal when compare 


Wi й 
by dallas data obtained in other 07 

Onal studies. 
ay 908 observation in the Del-Rey 
as directed primarily toward re- 


Spo. 
ns s * 
©з which could occur at any time 


and did not require the introduction of 
test conditions, it follows that poverty of 
environmental stimulation would not be 
expected to yield much evidence of re- 
tardation. The child left to his own de- 
vices on his back in his crib can bring his 
hand to his mouth, grasp his bed clothes, 
vocalize, observe his own hands, grasp his 
own hands, grasp his own foot, bring foot 
to mouth, etc. These are the items which 
were observed. One of the major findings 
of the Del-Rey study was that the un- 
tutored infant does do these things, and 
does them within the usual age range of 
home-reared babies. 

In regard to such responses it may be 
that the Creche babies are normal. The 
relevant facts can be discovered only by 
observers each spending full time observ- 
ing a few infants. If all Creche infants 
were to be observed it would necessitate 
the presence of many additional observers 
or caretakers. The reader is reminded 
that the Del-Rey investigation, involving 
only two infants, took a major part of the 
time of the two observers for one year. 
To devote one year, or even one month 
to observing each Creche subject cannot 
be proposed in an institution which has 
severe limitations of caretaker personnel. 
In contrast to the requirements of an ob- 
servational study of development, E 
testing time in the Creche study was only 
10 to 30 minutes per subject. 

If we cannot compare Del and Rey 
with the Creche babies in terms of sei 
vational data, it is likewise not pas * 
pare them in terms of test data. 
to 


estimate in retrospect 
ree of confi 


dence how Del 
and Rey would have scored e a 
. es during the first year on the Ga 
үз Scale. We arrive, therefore, at the 
И БУТЕ conclusion: It is bar 
with restricted learning oppor u 
normal on “observational 


to com 
is impossible 
with any deg 


items but re- 
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tarded on “test” items. It is believed that 
the latter, but not the former, are influ- 
enced by environmental limitations. If 
this is a correct interpretation, the Del- 
Rey study and the Creche study are two 
sides of the same coin. However, to estab- 
lish that this is the case appears to be a 
very difficult research assignment. 

We consider next the work of Spitz. 
The observations by Spitz which seem 
most closely related to the present study 
concern the institution called Foundling 
Home. Here, as at the Creche, there was 
a shortage of personnel. Although the 
mothers were present in the institution 
for several months, they seem to have had 
little contact with their children aside 
from breast-feeding them. Pinneau (9) 
points out that Spitz does not explain 
why this was the case. Despite the pres- 
ence of the mothers in the institution the 
adult-child ratio in the nursery is re- 
ported to be about 1 to 8, The children 
spent most of their time for many months 
on their backs in their Cribs, as did the 
Creche infants. At one point Spitz reports 
that a hollow worn in their mattresses 
restrained their activity. This, however, 
was definitely not true of the Creche in- 
fants. 

Since Spitz's studies have been exten- 
sively reviewed and criticized by Pinneau, 
only a limited amount of Space will be 
devoted to them here. Spitz used some 
form of the Hetzer-Wolf baby tests. 
There is no doubt that their standardiza- 
tion leaves much to be desired. Spitz re- 
ports scores for the Foundling Home 
group and a control group of 17 home- 
reared infants. In the second month both 
groups had mean DQ's between 190 and 
140. The private home group remained 
at that level but the mean of the Found- 
ling Home group dropped precipitously 
to 76 by the sixth month and to 72 by 
the end of the first. year. Spitz believes 


that this decline in DQ was due to the 
emotional consequences of separation 
from the mother, but Pinneau has 
pointed out that most of the decline took 
place prior to the prevalent age of sep- 
aration. Pinneau indicates further that at 
least some of the decline is probably due 
to inadequate test standardization. à 
We compare our data with those O 
Spitz with considerable hesitation” be- 
cause the two sets of data were obtained 
by tests whose comparability is unknown. 
In numerical terms the results of the two 
studies in the second half of the first year 
seem to agree fairly well, Spitz's msn 
for this period being about 74 and ours 
63. But the findings for the first half-year 
present some apparent differences. е 
subjects drop from a mean of 97 t° 4 
mean of 72 between the second and thir 1 
months, and drop only ten nage 
points thereafter. Spitz’s group can 
higher and declines for a longer perioc Ж 
Spitz’s data and ours agree in рае. 
that environmental conditions сап ч 
press infant test scores after the — 
month of life. We disagree with Spitz E 
regard to the interpretation of the cau " 
of the decline, He believes it to have on 
due, in the case of his subjects, to а ae 
of the emotional attachment to t! 
mother. This could not have been ШЕ 
cause of the decline of the Creche x 
fants. Since the conditions for the forma 


s à : scii m 
ton of an emotional tie to a specifi ji 
| h 


ivi " breacl 
dividual were never present, no we 


attachment could have occurred. ra- 
have noted above Pinneau's demum. 
tion that even Spitz's own data do ae 
Support his interpretation. We in tis- 
that Spitz’s data as well as ours are E e 
factorily interpreted in terms of — "à 
learning Opportunities. We suggest a 
àn analysis of the relationship € 
test items and the conditions prevail 
in the Foundling Home would rev 


INFANT DEVELOPMENT UNDER ENVIRONMENTAL HANDICAP 11 


that retardation could readily be ex- 
plained in terms of restriction of learning 
opportunities. But such restriction is not 
inherent in institutional care. Klacken- 
berg has recently presented a study (7) of 
infant development in a Swedish institu- 
Чоп, in which the adult-child ratio was 
1 to 2 or 3, in which no retardation was 
found. 


Discussion of the Creche Four- and 
Five-Year Olds 
We have no doubt that on many tests 
the Creche four- and five-year-olds (and 
also two- and three-year-olds) would be 
retarded, perhaps to a marked degree. 
We think this would be particularly true 
“ regard to tests involving more than a 
Very modest amount of language compre- 
hension and language usage. The lan- 
Вчаре handicap of institutional children 
With limited adult contact has been suffi- 
“lently demonstrated (8). 
© ү likely that on some performance 
below i Greche children also would Score 
ture таш norms. On the Healy Pic- 
the fn = m Board, for example, most of 
bie represented are outside the 
ше Sons of Creche children. We "d 
Yara the older Creche children are 
ire. ed on some tests, but we wish to 
ог ч whether retardation is general 
Vin lether it is related to specific en 
mental handicaps. 
йын chose the draw-a-man test, 
X cube test and the Porteus maze test 
mo it was thought that the Creche 
than ш might affect these tests less 
are m ler tests. So far as the Knox cubes 
low oncerned, it is difficult to imagine 
Завь сое сап deprive a child of the ех- 
touched of visually remembering үт 
sight г objects, except through loss c 
. So far as the Goodenough is con 


Cer 7 
ned, both human beings and two-di- 
m were 


the 


ensi 5 
Sional representations of the 


familiar to the subjects. They were also 
familiar with the idea of drawing and 
with the use of pencils. Knowledge of the 
use of pencils may also play a part in the 
Porteus maze test. It is uncertain what 
other experience may play a role in this 
test. 

The results show clearly that on these 
tests the Creche children approximated 
the performance of children in normal 
environments. In other words, the re- 
tardation which was found to exist be- 
tween 3 and 12 months of age did not 
produce a general and permanent intel- 
lectual deficit. It is possible for infants 
who have been retarded through limita- 
tions of experience at an early age level 
to perform normally, at least in some re- 
spects, at later age periods. The assump- 
tion that early retardation produces 
permanent retardation does not receive 
support from our data. 


Emotional and Personality Effects 

No doubt many readers would like to 
know the emotional and personality con- 
sequences of the Creche regime. So would 
we. But to the best of our knowledge no 
objective and standardized procedures 
with adequate norms are available which 
would enable us to compare the Creche 
infants with other groups of children in 
these respects. This is equally true of 


studies conducted earlier. 
ence of objective techniques, we can 
few impressions. The Creche in- 
oachable and were inter- 
ested in the tests, Very few testing sessions were 
postponed because of crying, dont n 
cause. There was very little shyness or fear Ls 
strangers, perhaps because each infant saw Severa 
ifferent adults. In the cribs there was very little 
d ing that did not scem attributable to 
e ie : discomfort. However, some of the older 
e бео ped automatisms such as arching 
ers ба, or hitting some part of the 
€ ү with the hand, which may have represented 
l 14 “stimulation hunger.” It was almost 
Е ОВЕ н ssible to get the infants over two 
oe ч B оре to smile by stroking their chins or 


In the abs 


only report a * 
fants were readily app™ 
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cheeks or by shaking them slightly. The older 
children, like the infants, were friendly and ap- 
proachable. However, such observations are not 
meant to imply that other Personality conse- 
quences could not be found if adequate tech- 
niques existed, 


SUMMARY AND CONCLUSIONS 


This study has been concerned with 
the development of children in an insti- 
tution in Beirut, Lebanon, called the 
Creche, in which “mothering” and all 
other forms of adult-child interaction are 
at a minimum because the institution is 
seriously understaffed. The children come 
to the institution shortly after birth and 
remain until six years of age. Contact 
with the mother ceases upon the child's 
entrance to the institution and contact 
with mother-substitutes is slight because 
the adult-child ratio is 1 to 10, 

Opportunity for developing infant 
skills through Practice is very slight. In 
the early months the infants are swad- 
dled. For many months the infant lies on 
his back, and is even fed in а supine 
Propped up, carried 
with the means of 
practicing many activities, 

Data on behavioral development were 
obtained by giving the Cattell infant 
scale to all infants between two and 
twelve months of age and the Good- 
enough draw-a-man test, the Knox cube 
test, and the Porteus maze test to all 
children between 4% and 6 years of age. 
Comparison data were available from 
American norms апа from certain groups 
of Lebanese subjects, 

It was found that in te 


rms of develop. 
mental quotients, the m 


was 63. In the tests given at the four- and 
five-year level, the mean scores were 
roughly go. 

Pall, interpretations of these data 
have been discussed at some length. КУ 
conclusions may be summarized as fol- 
lows: май]? 

1. It is uncertain whether the norma 
ity of behavior at two months shows ү 
Maturation plays a major role in ear M 
development, or whether operi 
limited as it was, provided the essen = 
requirements for learning the respons 
which were tested. "m 

2. The retardation prevailing moor 
three and twelve months of age seems н 
be due to lack of learning рейши 
in situations comparable to the же 
ations. It is possible that an Vii 
approach in the day-by-day ape a 
might reveal that some behaviors € 
oped normally. то 1085 

3. The infants did not underg br 
of an emotional attachment. I a 
nothing to suggest that emotional s epis 
or lack of mothering or other pss for 
arousing conditions, were responsib 
behavioral retardation. ths 0 

4- Retardation in the last 9 e 1р 
the first year to the extent of a iim J00" 
of 65 does not result in a general y! ла 
performance at , 10 to 6 years, even -1с(6 
the child remains in a relatively rest does 
environment, The study tele тш 
not support the doctrine of the is 
nency of early environmental effec" da 

5. It is believed that the objectiv a 
of other studies, as well as this pet 0 
be interpreted in terms of the cf infant 
Specific kinds of restrictions upon } 
learning. 
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I. INTRODUCTION 


ii ipe primary problem of the present 
ratae. | — to: determine whether a 
Sponse T significant gradient of re- 
warded engi of errors around re- 
removal ghe would remain after 
Probability.1 ^ Y guessing-sequence and 
directly =. dias factors by control groups 
Brouns. comparable to the experimental 
ps. 
аў qr reported by Thorndike (16, 
has receiy spread-of-effect” phenomenon 
Cation 7e ed substantial empirical verifi- 
ever, thas oe disagreement, how- 
iam eveloped about its theoretical 
the б aic 1 horndike. (16) regarded 
tition Керр decreasing intertrial repe- 
from errors with increasing distance 


а rewarded res : rong evi- 
dence d response as strong 


Tew, for the direct strengthening by 
ard of « | g 1 
td of S-R connections. He saw it as 


an © 
— “independent proof" of his law of 
iT | 
le st з for this study were collected during 
S Crs period of employment as Research 
at the Human Resources Res arch 
as fs en’ Ше Air Force Personnel and Trai 
Texas, ue Center), Lackland Air Force Base, 
Yerein ate РаПе and conclusions contained 
Nstrucd those of the author and are not to be 
Ment of as reflecting the views or the endorse 
Partial the Department of the Air Force. А 
tienga n of the study was given at the 
SSociatign SE of the American Psychological 
n in September, 1951. (Abstracted in 


Mer, P. 5 
А Sychologist, 1951, 6, 293.) 


hara, "m Ereatly indebted to Dr. Shinkuro 
the м his assistance in supervising most of 
asistan cal work, and to W. A. Hillix for his 

ce in the preparation of the manuscript. 


Iwa- 


ellect. More recent investigators have 
presented a wide variety of alternative 
interpretations. These range from the 
suggestion that the Thorndike effect is 
simply one special case of a more inclu- 
sive generalization-of-reinforcement prin- 
ciple (9) to the conclusion that the ob- 
tained gradients bear no essential rela- 
tion to reward but are artifacts of such 
conditions as number-guessing habits and 
probability bias in the response series 
3, 4, 14 15). 

The present study is an attempt to 


ide data on this problem with more 


prov 
e hitherto 


adequate controls than hav 
been used. The necessity for such con- 
trols is pointed up by the various studies 
which have shown that artifactual gradi- 
n be produced under certain con- 
ditions in the absence of reward. How- 
the gradients that have been ob- 
tained under conditions designed to test 
the guessing-sequence апа the proba- 
bility-bias hypotheses cannot be directly 
compared with those obtained under the 
conditions of reward following 
ition to learn. The evidence thus 
be used to support the 
hese alternative factors 
conditions, produce 
kinds of data cannot 


ents са 


ever, 


usual 
motiva 
far obtained may 
conclusion that tl 


can, under certain 


answer t E 
factors are necessary ОТ sufficient to ex; 


plain the typical Thorndike gradients 4 

obtained within different experimental _ 
Urse; 

frameworks, These results do, of egi ; 


e 
fe 


N 


gradients; but these 
8 he question as to whether these A 
< 
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throw suspicion on the Thorndike inter- 
pretation and point to the need for direct, 
controlled comparisons of the sort at- 
tempted in the present investigation. 
This problem is discussed at greater 
iength in the recent review of the spread- 
of-effect literature (8). 


П. METHOD 
A. Subjects 

Four groups of 300 basic trainees at 
Lackland Air Force Base were used. Two 
of these were experimental groups, E and 
EU; two were control groups, C and 
CRB. Data for the first two trials from 
an additional 74 Ss were also used for 
certain of the analyses, since these Ss were 
treated exactly like those of Group С, 
who had been tested earlier. This gave a 
pooled Group C of 374 Ss and a total of 
1,274 Ss in the study, 

All of the Ss were basic trainees from 
flights at Lackland Air Force Base be- 
tween October 1950 and August 1951. 
They were all tested individually by the 
same airman, who was thoroughly trained 
аз to the testing procedure but naive as 
to the purpose of the Study. Ss were made 
available from the various flights accord- 
ing to the assignments worked out by the 
üirman in charge of their distribution 
and were then placed in the various 
groups according to а mechanica] System 
of rotation which was Prepared in aq. 
vance. 

Throughout the Study it was found 
necessary to discard only 20 Ss. Most of 
these eliminations were for 


some mis. 
understanding of the instructions or ex- 
perimental error in test administration, 


These cases were discarded Ь 


у E before 
the records were analyzed. 


B. Apparatus 


A SAM complex coordination 


Test, 
Form E (11), was modified in 


accordance 
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with the requirements of this study. = 
the aileron lights were used. A plywood 
mask screened off the elevator and rudder 
lights to avoid distraction of the S The 
resulting display is shown in Fig. 1 
which is a frontal view of the apparatus 
(not drawn to exact scale). 


Movements of the stick from the a 
ing position to right or left pa! e oh 
the flashing on of the appropriate green rep- 
in the lower row of lights. The middle Ыш nut 
resenting the resting position of the ai ена for 
used, This left a total of 12 lights ak е E 
settings, 6 to the right and 6 to the un lights 
central resting position. The top row of dm were 
Was used only as a signal. rA 
flashed on, as a group, for the "right" signa acd; 

The normal tension on the stick piu its 
was instructed to allow the stick to о аде a 
central resting position as soon as he hac r wrong 
теѕропѕе and had been given the right О 
signal. :croswitclt 

ow rudder bar was fitted with a пш e 
and wired into the control panel to make [ The $ 
knowledge of results after cach рин ШЕ 
Was instructed to push the rudder a light set 
right foot immediately after making Ph the 
ting. Flashing on of all of the red lig д 
upper row—the signal for right—or bac 
à buzzer—the signal for wrong—then OC 


long as the bar was depressed. са for E, 
^ simple control panel was à ru display 
Who sat at a table immediately behin 


^ ‘er 
lights cor" 


able t0 5 
е as they 


ent in ОР” 


Panel. This consisted of a row of 1 
sponding to the 12 light settings avi | 
and enabled E to observe 5% respons 
were made. A master control switch, šil t or 
cration, enabled Е to arrange the an 
Wrong signal in advance of cach response 


ved use 
This apparatus made it possible easel 
all the data. In almost all the ae Sa 
thus far reported on this probier on 
sponse strength has been measure of 1€ 
by the percentage of apes mé 
Sponses (cf, 8). This all-or-none аш” 
Measure is more difficult to treat ait ir 
cally and less sensitive than a тез oft 
Which tendencies toward spese suc 
5ponse can be quantitatively ee, 
а quantitative scaling was made ия in 
in this Study by using the variat ma 
linear light units, between settings asf 
On successive trials. Use of this Me’ 


y 
М 
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A 


was а 

ки de assumption that the 
from one унад would be less varied 
repetition ч: ко the HERE uM, EAN 

The ia ч чө limiting case. 
nhize аы “ч also bea, rs to thi 
sponse ban м» pre-experimental re- 
been ee of the sort which have 
studies and e dn the number-guessing 
for the EE rich have formed the basis 
the "зу; LEHENGE explanation of 
ike gradients. 


C. Experimental Design 


i, Ge 
eneral Testi 
Testing Procedure? 


Instr i А 
groups тр for both experimental 
group hs sian ك‎ the standard control 
told that fuse identical. Briefly, Ss were 
еее» ch ir ‘kde was to discover and 
to light the е леш sequence in which 
the settings LE lights in the row; that 
er, | Fes be called out by num- 
gh 12; that they would have 


only 

on 4 

they - chance on each setting; that 
tri e simply to guess for the first 


al si 
падва р пеле had been deter- 
and йыл: arbitrary, random manner, 
Were ea sequences of responses 
they could ore to be avoided; and that 
the bezinr Not expect to do very well at 
the task Onn because of the difficulty of 
then dem peration of the apparatus was 
le шене UNUM with emphasis on 
аг, Right es of the stick and rudder 
onstrated Tos wrong signals were dem- 
his ри he 5 was required to tell £, 
the prog n words, the essential parts of 
Secene: Any aspects that seemed 


Э ý The " 
tions ee instructions used for all condi- 
ulations sige deposited, along with certain 
т Nation р data, with the American Docu- 
qa DI Reon Order Document No. 5281 
ine. ication гиц Publications Project, Photo- 
‘on 95 ervice, Library of Congress, Wash- 
; aoe C., remitting in advance $1-75 for 
РО! осоре rofilm copies or $2.50 for 6 x Sin. 
oduplic. Make checks payable to Chief, 
Plication Service, Library of Congress- 


tabu] 


Fic. 1, Drawing of the apparatus. 


to be unclear were then reviewed by Ё. 


After E was satisfied that $ understood the gen- 
eral procedure to be followed he began the ex- 
eriment, Light settings were called out as fol- 
lows: "Setting No. 1," "Setting No. 2," etc. Ss 
were allowed to proceed at their own pace, al- 
though if an 5 was markedly slow he was cau- 
tioned to speed up his responses. The trials of 
12 responses each averaged approximately go 
sec. A fixed intertrial interval of go sec. was 


used. 


E was instructed to watch for any tendency to- 


wards systematic response sequences which 5 
might show in spite of the strong caution in the 
instructions. Preliminary experimentation indi- 
ed that the most common examples of such 
e repetition of a single light 
position, with the apparent intention of elimi- 
nating it, and the progressive movement by one 
or more light units in one direction along the 
row of lights. 5s showing such obvious systematic 
sequences were cautioned by Æ when he was 
reasonably sure of the sequence, ordinarily after 
the third response in the series, This caution 
coming after the emphasis on this point in the 
instructions was 50 successful that only seven Ss 
had to be discarded for having persisted in mak- 
ing such systematic sequences of responses. These 
were discarded upon examination of the records 


rior to any scoring or analysis. 
tructions given to the EU Ss 


The special ins 
first trial, and the entirely dif- 


at the end of the 
ctions given to the CRB group 


ferent set of instru 
will be found along with the standard instruc- 


tions (see footnote 3). 


cat 
sequences were th 
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Before they left the testing room, Ss were re- 
quested to refrain from discussing the experi- 
ment, As an additional precaution against the 
giving of information, Ss who had been tested 
were kept in a separate waiting room, and in a 
different building when possible, from those who 
were yet to be tested. Since Ss at each testing 
session were drawn from different flights, be- 
tween which contact was minimal, this possible 
source of passing information was disregarded. 


з. Experimental Group E 

One subgroup of 150 Ss was rewarded 
on the second and the eleventh responses 
throughout the five trials (E-2); the other 
subgroup was rewarded on the sixth re- 
sponse (E-6). Reward in these stimulus 
positions was given regardless of the re- 
sponse made, This procedure permitted 
measurement of error strength for the 
four stimulus positions before response 
11 and the four stimulus positions after 
response 2; and for the four stimulus 
positions before and after response 6. 
The two subgroups w 
possible differences ass 
position factors, 

The primary concern was with the first 
two trials. However, additional prear- 
ranged rewards were given on the third, 
fourth, and fifth trials. This was chiefly 
to prevent suspicion and to keep up 
morale in the subject population in case 
there were any discussions of the proce. 
dure. Little use was made of these addi- 
tional data. 


ere used to detect 
ociated with serial 


3. Experimental Uncertainty Group EU 

This was an added experimental group 
designed to determine the effect of the 
so-called “uncertainty” factor (cl. 8). 
These Ss were treated exactly like the 
standard experimental (E) Ss throughout 
the instructions and the first trial, At 
the end of the first trial, how 
were told that they would be informed 
during the second trial whenever the 
light setting they were about to make 


ever, they 
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was one of those that they had responded 
to correctly on the first trial. They jua 
then so informed, as the акр" 
stimulus was given, but were not told e 
setting which had been correct on E 
first trial. It was believed that thus = 
ing the 5 as to where in the pepon 
series he had been right on the first tr? 
would effectively remove any pee 
he might have on this point and y 
eliminate deliberate repetition of 
sponses near the rewarded-response po 
tion. Я 

The EU Ss were divided into two 
groups of 150 each according to ү я Ss 
Поп of the rewarded responses. Ё cath 
were rewarded in the second and midi 
response position; EU-6 5s were or 
in the sixth response position. ЫЕ 
the E Ss, they were given only two 


4. Control Group C 
1 1 gg. who 


This group consisted of 300 vo 


~et LV 
were given no rewards on the Lee 
trials. Records from these two tri2 > on 
stituted the basic control data for sults 
parison with the experimental N were 
On the third and fourth trials an anc 
arbitrarily rewarded on the ae the 
eleventh responses, making е enta 
accumulation of additional expert re 
data. One or two further ain 
wards were given on the fifth tria оппо 

The additional 74 Ss from à "arit 
8roup originally designed for а ejt ne 
Purpose were treated exactly hen 2 
ordinary controls throughout en er 
therefore could be pooled with t1 trial! 
the important analyses based ОП stand 
1 and 2. They differed from E of а? 
ard controls only in the inclusion mak 
additional two nonrewarded a the 
ing a total of four such trials, eal А 
first arbitrary rewards were 81% p? 
the second and eleventh stimulus 
tions. 


and 


si 
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5. Response-Bias Control Group CRB 


wae was designed to control for 
iin af d due to the posi- 
helped to шша response. These Ss 
cul а test the relevance of Smith's 
ا‎ that gradients may be 
tition af " Rais followed the repe- 
ies : correct response, by the restric- 
баши expanse range that occurs be- 
immediate «à tendency lor 35 to avoid 
also sir ENS of a response. Л hey 
lilases fa | М for any pre-experimental 
emphasize Y tésponse series, of the kind 
Suck biases oy Jenkins and Sheffield (4) 
ШЕН yn E to be considered even 
freedom task was designed for relative 
CRB rom them. 
with d ише individually matched 
the partici pipe E Ss on the basis of 
latter in dá ad light settings made by the 
during th eins rewarded positions 
forced ү E two trials. They were 
of adjust; ae gene in this way by means 
immediately pointers, which were set 
Other Sinai before each trial. In all 
owed ^s pma positions they were al- 
Or the e as they pleased except 
Systematic ie caution against the use of 
this Cii. сы. In order to make 
Were told ix reasonable to the Ss they 
ing йн m operating checks were be- 
Wrong leg the apparatus. Right and 
subgroups чүзү not involved. Tes 
$ B-6) thn 150 Ss each (CRB-2 and 
b used, corresponding to the 


E о 
conditi » у, 
trials. itions, They were given only ? 


III. RESULTS 


ny is сар the results, it will be neces 
еги Шы: Somna the meaning of several 

Osition” ME need repeatedly. “stimulus 

sei ae аша setting number” will be 
ition erehangeably to refer to serial 
Sition i a trial. That is, “stimulus 

refers to that position within 


a trial at which E called out “Setting 
number 6” and $ subsequently made his 
sixth response in the series of 12 re- 
sponses which constituted a "trial." 
"Light" or "light setting" refers to the 
spatial location of the particular response 
S made for any of the stimulus positions 
within a trial. $ could make a light set- 
ting from 1 through 12, since there were 
12 lights on the panel. 

In the experimental groups “error 
position” generally refers to the stimulus 
position relative to the reinforced stimu- 
lus position, Thus the “+1” error posi- 
tion is the stimulus position immediately 
after the reinforced stimulus position; 
“Lı” js the position immediately 
etc. Responses made in the criti- 
arded) stimulus positions (second 
enth, or sixth) are referred to as 


the 
before, 
cal (rew 
and elev 
“key responses.” 
In the control groups, for purposes of 
n, error strength was measured 
before and after certain nonrewarded 
responses. These responses are therefore 
also called “key responses,” regardless of 
their stimulus position. Other errors, 
before and after such key responses, are 
referred to as “44,” etc, as in the case 
of the experimental groups. 
As earlier described, response strength 
was measured by the variation, in linear 
light settings made 


light units, between 
the first two trials at a particular 
on. For example, if 5 re 


with the sixth light on trial 1 


rth stimulus position, and with 


ght on trial 2 in the fourth 


he was 
e of 3 (6 minus 3). 


сотрагіѕо 


оп 
stimulus positi 
sponded 
in the fou 
the third li 
stimulus position, 

error difference scor 
т—Ттїа1 2 Analysis 


assigned an 


A. Trial 


1. All Ss 
This section рї 

300 Ss of each 

after-gradients, 


esents the data from the 
major group. Marked 
but no fore-gradients, 
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Fic. 2. Error-difference gradients in Group E. 


were found for both experimental groups. 
These are shown in Fig. 2 and 3. Applica- 
tion of a trend analysis, using the tech- 
nique of analysis of variance (5), showed 
that both of the experimental after- 
gradients were significant at well beyond 
the .o1 level of confidence. The Fs were 
5-46 and 7.22 for E and EU groups re- 
spectively; 3.80 is required for 3 and 897 
df at the .о1 level, 

The reversed fore-gradient for Group 
EU in Fig. 3, which was in a direction 
opposite to the prediction, was not sig- 
nificant (F = 2.47; 2.61 required at the 
05 level for 3 and 897 df). F was not 
calculated for the other curve, since there 
was no indication of a fore-gradient in 
the raw score means, 

The corresponding curves for the two 
control groups are shown in Fig. 
5. No evidence of any gradient was 
in Group C. The gradient in Group 
CRB was very nearly statistically sig- 
nificant (F = 2.58; 9.61 required at the 
-05 level for $ and 897 df). However, this 
is not as critical an analysis for the CRB 
Group as the analysis of the records of 
Ss who repeated the key response (the 
response made in the same stimulus posi- 


4 and 
found 


* Wherever necessary, Hartley's (2) Suggestion 
for testing homogeneity of variance was applied 
In no case was the ratio large eno 


ugh to cause 
concern. Norton (12; see also 5) has 


that deviations from homogeneity с 
losses іп accuracy than has been со 
posed. 


pointed out 
ause smaller 
mmonly sup- 
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tion as the rewarded response for ш 
matched E-group Ss), thus providing an 
“anchor” for their response biases. -— 

The after-gradients were statistica е, 
significant in three of the four m 
mental subgroups but in none of the : Y 
trol subgroups. These data are availab 
from the ADI (sce footnote 3). 


2. Repetition of Key Responses " 
The data were next analyzed acggrding 
to whether or not the key (генийден) m 
sponse was repeated on the second mu 
This analysis showed that the after8 p 
ent in the experimental groups np 
pendent upon the repetition or This 
repetition of the rewarded p erede ^ 
is clearly shown by the three e" ae 
Fig. 6. Here the зоо Ss of Group on 
been divided into three subgroups in 
the basis of the amount of ipea d 
the. rewarded response from tria figure 
trial 2. It is apparent from the poth 
that gradients occurred following ?; 


ied 

adn varie 

exactly repeated and minimally isti- 
Thes re sti 

rewarded responses. These we leve 


cally significant at beyond the SE 3.83 
of confidence (F's of 4.93 and ш ‘for 3 
and 3.94 required at the .o1 ). 
апа 543, 3 and 135 df, respective ү x 
gradient occurred following а y d re 
tensive variation in the seva" ын i 
sponse. Again, there were no forc 
ents, 


ак 
ae ra bre 
Similar results were found for 


450 


MEAN ERROR DIFFERENCE 


о 
BEFORE REINFORCE! 
RESPONSE 


RESPONSE POSITION Group 
Fic. з. Error-difference gradients in 
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БО it invemion from position 3 to 
Eust кд emn пы. oi 
sponses, Thi к mean rewarded re- 
yond the is was significant at well be- 
uired for ют level (Е = 6.93; 3.83 Te 
Vatlations Pan also found following 
warded 5 one light unit in the re- 
Mr unions eit although this was not 
у E p. qoe: (Е = 1.27; 2.65 Te- 
No us 5 level for 3 and 177 df). 
уймайоп 2H was found following greater 
iion: s (2 to 11 units) in the rewarded 
se. 
gd ed data for these. breakdowns 
Positions pouring data for the fore- 
e e er the ADI (see 
lowing > he predicted gradients fol- 
curred in EO rewarded responses oc 
subgroups : of the four experimental 
cally si s although they were staust- 
and nb саш only for subgroups E-2 
cant gradi The fact that a highly signifi- 
"Toup Ha gas (Р= .01) was found for 
that were blowing rewarded responses 
that the мош by опе unit suggests 
an early Lin ients were not peculiar to 
rhe Til position. 
lowing ee response strength fol- 
the two за of the key response in 
3. The x» p ol groups are shown 1n Fig. 
the С on 5s represented are those from 
Cither Dur. of goo Ss who repeated 
second or their sixth T€ 
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Fic. 5. Error-difference gradients in Group CRB. 


sponse. The 182 CRB Ss are those who 
were matched with E Ss who repeated 
either their second or their sixth re- 


sponse. 

No control gradients occurred, An 
analysis of variance for the CRB curve, 
which gave a slight indication of a gradi- 
ent, showed that it was not significantly 
different from a horizontal straight line 


(Е = 1.93: 2.62 required at .05 level for 


запа 543 df). 


Although the gradient for the 182 E 


Ss who repeated the rewarded response 


was highly significant (beyond .01 level) 
and there was no gradient for the 52 С 
repe itions, the direct 


aters at the same post 
comparison of the slopes failed to yield 
a significant © 


lifference (F < 1-00). The 
same was true for the comparison of the 
182 E Ss with the 182 CRB Ss who re- 
ted at the same 


positions (Е = 1.745 


реа 


MEAN ERROR DIFFERENCE 
* 
© 


SE POSITION 
of error difference in 
п of degree of repeti- 
response. 


AFTER- RESPON! 


Fic. 6. After-gradients 


Group E as а functio 
tion of rewarded 


MELVIN 
8 


MEAN ERROR DIFFERENCE 
$ 


3 z 


AFTER-RESPONSE POSITIONS 


Fic. 7. After-gradients of error difference for 
control groups following repeated 
nonrewarded responses, 


2.61 required at the :05 lev 
1,086 df). 

The CRB Ss who exactly repeated 
their key responses are those to w 
the probability-bi 
The gradient for 
proach significanc 


el for 3 and 


hom 


2 and 6 only. They thus pr 


response in the two experimental groups. 

A control curve based upon a larger 
sample was constructed for gradients fol- 
lowing the repetitions of resp 
of the first eight stimulus ро: 
use of such repetitions seems 
fied by the finding, noted above, that the 
experimental gradients occurred in both 
the second and the sixth Positions, The 
generality of the experimental results 
over the early and middle Serial positions 
was thus strongly suggested, 

A total of 201 Ss who repeated at least 
one response within response Positions 1 
through 8 was obtained from the 374 C 
Ss. Repetitions of responses in the last 
four positions w 


onses in any 
sitions, The 
to be justi- 


Y response. Since 
al analysis each § 
Once, it was de. 


for purposes of statistic 
could be counted only 
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cided that the first repetition of each 5 
would be analyzed. Results for all of the 
repetitions were also computed, po^ 
ever, and are available from the xs n 
(see footnote 3). The e ape 
values for this group of 201 C Ss mes _ 
some tendency toward an after-gracie е 
because of the low error-difference жо 
for the first error following the om 
response. However, the xia gea a 
did not approach statistical gener 
(R= туб. 6 required for the 05 f all 
for $ and боо df). The img 
of the repetitions of the 201 T uds 
repeated responses within the за rially 
response positions did not mate 
change the results. at the 

As additional checks, in order tha tly 
first error score be tested independen i 
of the trend test for a gradient, дй 
t tests were run between the mean bis for 
tion score for the first error and я 0 
each of the other three errors. AE ab 
the t's were significant (values in with 
1.75, and .59 for the comparisons wn 
second, third, and fourth error pw a 
respectively; 1.97 required for t? 
level). . T 

For purposes of comparison sim 
were run for the two experimental » 
after repeaters. The difference d fourth 
Variation scores between first and d 1€ 
errors following repeated rewal to 
Sponses was found in each rand me 
highly significant (t for E = Iae 
EU = 3-76; 2.62 required for .01 » is 

A direct comparison of the s и 
the experimental and the above ny ig 
gradient again failed to show апу pe 
nificant difference (F= 1.59; 7 ар: 
quired at .o5 level for 3 and 1,143 


lar % 


roup* 
mean 


В. Analyses Beyond Trial ? m 


ro 
Р ined f 
These data were mainly obtain qav 


К Ж 2 апб e, 
the use of reward in positions 2 55 


o 
or 6, on trials 3, 4, and s for the 3° 


———— —————' 
> 
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They were thus necessarily somewhat con- 
oe by interference from response 
‘thet хо over from trials i and э. 
ADI ji Oss results are available from the 
see footnote 3). 
bo igh no gradient between trials 
Î т: UR reward only on the second 
There e dia not result ina gradient. 
епнге ^x » ery good gradient of error 
ч жш : trials 8 and 4. Continuing 
for el i aia, for trials 4 and 5, and 
group 4 n. on trials ? and 3 in the E 
cw icate that the gradients con- 
Ma ы the second rewarded trial. 
owing p e statistical tests were used, 
йы, tien the relative impurity of the 
gradients ; consistency with which the 
appeared is of some interest. 


C. T 
ests of Alternative Explanations 


ver number of additional analyses 
Planatior - test various alternative ex: 
gradi 15 of the experimental after- 
ents, 

"^h ERA hypothesis concerned only 
connection b strengthening of an S-R 
He assum i contiguity with reward. 
tablished that this strengthening, es- 
2€ more Me. the first trial, caused Ss to 
to the sa ‘ ely to repeat the same error 
Was ШОО, stimulus when the stimulus 
t might ‘ei again on the second trial. 
Nstratio s argued that the simple dem- 
mental ien » differences between experi- 
to км c control gradients is sufficient 
the соп че Thorndike's hypothesis, since 
Büessing. c pu eliminate the и 
YWpotheses lence and probability-b1as 
he it is possible that reward acts 
“ ashion other than the "spread 
(17). ee postulated by Thorndike 
atrial маан ти create a stronger 10- 
On the р © ationship between responses 
tria] ah trial, and also stronger intra- 
ationships on the second trial. 


This would result in a stronger apparent 
intertrial relationship between first er- 
rors, yet would not be the kind of direct 
S-R strengthening with which Thorndike 
was concerned. For example, if the key 
response were the seventh light, and 
reward increased the probability that the 
following response would be the eighth 
light, then the probability that the er- 
ror difference score would be zero (S re- 
sponding with eighth light at the first 
after-error position on both trials) would 
be increased, independently of any actual 
strengthening of the S-R connection be- 
tween environmental stimuli and the 
choice of the eighth light. 

The analyses reported here were run 
to see whether an increase in intratrial 
relationships between responses actually 
urred as a side-effect of reward; and, 
this kind of response bias 
all of the ob- 


occ 
if so, whether 
was sufficient to account for 
served intertrial strengthening. Only 
gradients following repetition of the key 
response are considered, since only in 
that case were significant gradients found. 
These analyses should therefore give 
some information about the degree of 
confidence that could be accorded the 
conclusion that the after-gradients were 
not entirely artifactual. 


1. The Alternation Hypothesis 

If Ss tended to respond first to one side, then 
to another, in this apparatus, gradients could be 
created artifactually by this tendency. The key 
response if repeated would serve as an anchor 


oint on the same side on both trials. Alternation 
of the next respor 


nse to the opposite side on each 
trial would then result in a lowered ш 
difference score for that response. This fol es 
from the fact that if the response in any pec 
osition Was made to the same 28 оп oth 
trials, the maximum possible error dil erence was 
si th response was made to different sides on 
E nd jals, the maximum possible difference 
paeem the alternation tendency could 
*» hy the experimental gradients fol- 
ions were stronger than the con- 
alternation tendency was stronger 


al group. 


was 11. 
not explain w^ 
lowing repetiti 
trol, unless the 
in the experiment 
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TABLE 1 


PERCENTAGE oF ALTERNATION AND MEAN 
Ennon-DIFFERENCE SCORES AFTER 
REPEATED КЕҮ RESPONSES 


Error 


Group N Trial: Trial 2 Differ- 
ences 
E-2 85 77-6°* 2.1 2.55 
E-6 97 577 55-7 3-37 
EU-2 92 68.5 68.5 3.01 
EU-6 128 78:0" 65.6 2.82 
(е; 201 59.7 57.2 3.24 
CRB-2 85 80.0** 63.5 3-41 
CRB-6 оу 80.4** 69.1* 3:05 


* Significant at .05 level, 
Control value. 


** Significant at .or level. 


compared with 


Table 1 shows this analysis for the first errors, 
The first chance repetition at any position from 
1 through 8 was used for the C Broup (compar- 
able data for all Ss are available from the ADI; 
see footnote 9). 

Two of the four differences between experi- 
mental and control Stoups were significant at the 
+01 level on trial 1, and both of the differences 
between the CRB and C groups were significant 
at the same level, On trial 2, however, none of 
the experimental percentages were significantly 
higher than the corresponding contro] percent- 
ages, 

The low alternation Percentages for E-6 $5 
corresponded to weak error reinforcements, but 
the highest alternation percentages, in the CRB 
group, also corresponded to weak error reinforce- 
ments. Thus alternation alo; 


A ne was apparently 
not sufficient to produce gradients in these 
groups. The last column of Table 1 shows these 


mean first-error difference scores, 


Since these analyses were performed exclusive. 
ly on Ss who repeated the key Tesponse, concur. 
rence of either alternation (A) or nonalternation 
(N) on the two trials resulted in first after-errors 
on the same side of the central-rest position, Re- 
versal of alternation or nonalternation condition 
on the second trial resulted in first after-errors on 
opposite sides of central rest, Accordingly, the 
Ss were divided into four classes on the basis of 
their alternation or nonalternation on trials 1 
and 2. These were AA, NN, AN, and NA. The 
first letter designates whether or not S alter. 
nated sides from key response to first error on 
trial 1, the second letter on trial 2. For example, 
a 5 who made the key Tesponse on the left side 
on trial one, then the first error to the right on 
the same trial, would be classified with first letter 
А. As a repeater, he necessarily repeated the key 
Tesponse to the left on the second trial. Now, if 
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he responded to the right on the trial 2 firs 
error, he was classified in АА. iena 

The gradients should not have Bes pe 
nificantly within any of the alternatior vat S 
alternation gave rise to the ашан 1 i. 
once the groups have been equalized pes 
to alternation, the experimental ees 
should not give rise to any significant c | ДААП 

Table 2 shows this breakdown. сна E hs 
NN were combined on the basis of the m m^ 
the response tendency was consistent on 


sig- 


* the 
А — s 10 
trials, resulting in first-error rapon çes, the 
same side on both trials. In both 


ecrcased. 
» decreas 
error-difference score would tend to ue к for 
А L test was run between the шеш асил! 
Сапа E groups, and the diflerence was S18 


ses AA 
е EU classes / 
at the .o2 level. However, when El dilference 


at for 

score for EU was exactly the same as ae f 

Classes AN and NA were similarly poo j level, 
t for C versus EU was significant at the inh 
while that for C versus E was not signi sas con- 

Thus, when the alternation tendency outlined 
trolled by classifying the responses срегїтеп!а 
above, two of the four similar GNE difference 
groups showed significantly. smaller these four 
Scores after reward. Considering only 
comparisons, the chances of Betting 4 ce is 
P-values as small as even .о by сане t 
015. This calculation gives some int a 
there was still a difference between iUt 
the alternation tendency was sn by 

The same type of analysis into a was 
sponse tendencies on trials 1 and "in s 
formed for the CRB group, as shown 


тетет 
ncs A re diffe 

No statistical tests were made of tl nection ү, 

lilierence 


hat 


ps a t 


re- 


aces 


sting 
roup 


инеге: 


TABLE 2 T 
ICE ORE 
MEAN First-Error Dirr IRURE за. 
REPEATED Key RESPONSES 
ALTERNATION CLASSES —— <4 


Different sides 


JER 
AFTË 


Same sides, 


G АМ& Р 
ор p АА& „ N "NA 
N NN А 
? тад, 
Е 104 1.29 2.51* 77 5-30 2.25 
ЕП I41 1.69  .oo0 81 er 
С (т-тт) 148 1.69 92 {лю 
CRB LIS: т.53 op Pu 


` E and EU tested against C. 
* Significant at -03 level. 
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TABLE 3 
CENTAGE OF RESPONSES TO SAME AND Dir- 
ERENT SIDES AFTER REPEATED KEY RESPONSES 


Group 
Item z - 
— Е EL (1-8) (1-11) СКВ 
Total V 3 = == z = 


% Same Sides 57 
(ЛА NN) 7 


% Different 
sides —À ook ae! s 
(AN & NA) .3 36.8 30-3 38:0 36.8 


з ‚иг for any of the other experimental 
ot i. s p^ also greater than the control value 
down " lowever, when the group was broken 
EG nae components AA, AN, etc., none of the 
ктт were greater than those of the corre- 
alter d contro] groups. In other words, the 
eney Be TenHeney worked against the tend- 
Tlie f т errors to be reinforced in this group. 
МЕҢ the that the over-all score for Е-б was 
that i ‘зап for C is thus explained by the fact 
аы Percentage of the group which either 
the ба ed Jr did not alternate consistently on 
small 8s two trials (АА or NN) was significantly 
Pi сг than in the control group (2 = 689. 
CS ee for 1 df). 

PAL oly ings up the possibility that 
tal eau following, repetitions in the 
ber of pa might be explained by a E 
Side MUS ene Ss responding 10 the 
Would h the first crror on trials 1 and 2. Thi 
Such artificially reduce the difference pore 
nimbe tendency would be shown by a lm 
ine ; r of experimental than control Ss арреа 

en classes AA and NN. Table 3 shows the 
on of Ss in cach group who responded 
trials. same and to different sides ОП the 


the over-all 
experimen- 
larger num- 


am over-all difference among the proportione 
Was 1 EU, and C groups in classes AA and № 
fies а significant (x? < 1.00 for 1 df. This qo 
ош lat one might conclude from а gross 
an оп of the table: there does not scem to = 
apend ал. difference in the tendency to е 
n to the same side on both trials with s 
amo error response. The over-all i S 
Micra groups are not explicable in terms О! а 
$аг nra] numbers of Ss who responded to 
me side on both trials. 


2. The Central-Rest Tendency 


m Even though it did not appear likely that шее 
545 any group difference in the tendency (0 ма 
Pond to the same side with the first errors О! 
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the two trials, there remained a possibility that 
the reward affected the distribution of responses. 
A tendency was noted for Ss to restrict their 
response range following reward; this restriction 
would reduce the range of difference scores and 
might therefore have produced an artificial gra- 
dient. 

In order to test for this possible central-rest 
tendency, responses were arbitrarily divided into 
“central’ locations (light settings 4 through 9) 
and “peripheral” locations (light settings 1 
through 3 and 10 through 12). 

Table 4 shows the percentages of first errors 
following repetitions that were in central loca- 
tions. Three out of four obtained experimental 
values were smaller than the control values on 
trial 1; EU-2 was significantly smaller. In spite 
of the fact that E-6 Ss had a significantly larger 
percentage in central locations, they did not show 
a significant error-difference gradient. 

On trial 2, only EU-2 had a significantly larger 
value than the control. This shift from signifi- 
cantly smaller to significantly larger in group 
EU-2 was paralleled to some extent by a corre- 
sponding shift in EU-6, although the differences 
were not significant for this group. Altogether, 
it does not seem that the difference in percen- 
tages in central locations could have produced 
the gradients. А 

Further analyses were then run in order to 
compare the difference-score gradients following 
repetitions of central key responses M ш 
gradients following repetitions of perip keral key 
responses. Central responses were again 1 to 3 
units from the central-rest position, peripheral 
responses 4 to 6 units away. Table 5 shows these 


i ir P-values. 
radients and their P-va | Р 
There were significant gradients following the 
repetitions of a central key response in the ex- 


TABLE 4 
AL First-ERROR RE- 


ЕЕСЕХ 3 or CENTR 
Pence TER REPEATED Key RESPONSES 
uc O 15 Trial 2 
8 55-3 63.5° 
Ee E Bord 64.0% 
ЕР 92 44.6% 76.1 
Du 128 50.0 67.2 
5 201 56-7, 59-7 
CRB-2 85 41.2 $2.9 
CRB-6 97 532 51.5 


tly different from C value at .05 


nt from C value at .or И 
/ 


er cent at .о$ of 


a Significan 


1 cantly differe! 


el. 
Signin 
evel., 
c Sig 
zel. 
se Significant 
level. 


nificantly different from 5° p 


y different from 


go per cent at jor کک‎ 
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TABLE $ 


: SD PERIPHERAL 
MEAN EnROR-DIFFERENCE Scores AFTER CENTRAL AND PERIPHE 
“aan REPEATED Key RESPONSES 


After-position 


á ш 
нар а I 2 3 4 
.o1 
7 .o8 111 4.30 
Central 108 QE 3 А A 
Peripheral 74 3.32 3.43 3.32 3.45 И eS 
J ol 

SE run 135 2.69 3.70 4.10 3 à 
Peripheral 82 3.26 3:23 3.68 1.0. & 

C (0 & 1) эж, 
Central 60 4.12 3-78 3.68 4 38 s 
Peripheral 22 3.09 2.45 4.00 3 27 _ چیے‎ 

CRB - - 
Central 108 3.44 3.59 3285 3: = 
Peripheral 74 2.95 3.82 3.31 ы is E- 


* In order to increase the smal 
Score of one were included. 


perimental groups, but no other significant gra- 
dients. There was an interesting tendency for 
first errors after peripheral key 
stronger than after cen 
control group, This is 


group. 
checked showed a 
with position five 
tral error (data are 


; See footnote 3). Thus this 
effect in the control Broup does not appear to 


have depended on the seria] position in which 
the key response occurred, 
Since there were these diff 
gradients following central 
responses, it is not true that 
in response range following any reward is ade- 
quate to explain the gradients. The remaining 
possibility is that there was a gradient of restric- 
tion of response range following Teward of a 
central response, but not of a Peripheral re. 
sponse, 
This possibility was also checked, A different 
kind of gradient was calculated. This was a gra. 
dient of distances of the Successive error re- 
sponses from the central-rest Position. The abso- 
lute distance from central Test, irrespective of 
direction, was used for this analysis. The error- 
difference score would tend to be reduced if re- 
sponses were near the resting position, irrespec. 
tive of whether the Tesponses were to the left or 
right. 


erences between the 
and peripheral ke 
a simple Testriction 


Е аре чт ss with a diffe 
1 N for control repetition in positions 2 and 6, Ss wit 


oe rence 


t < were 
T radients W 
Table 6 shows that central-rest gradi 


hich hac 

found in the two central subgroups, d was 
significant error-difference ace roup fol- 
also a significant gradient in the E tus gradient 
lowing peripheral key responses, but he response 
was in a reversed direction; that i А following 
Tange was less restricted инша 2 from it 
the key response than it was ШИ йс tend- 
These results suggest that there мағ about the 
ency to make the first after-error at телата! 
Same distance from the center as the 
response. et consi 

га general, the results were pues vat 
With a hypothesis that the — UR response 
duced by a gradient in distance or possi 
from the central-rest position, It is 1 [ег or nO 
evaluate from this analysis alone bes га 
the central-rest tendency did ata tl 
the gradients. In the preceding analy f res 
cern has been with the distribution © ucstion i 
following reward. There was still a 9 for error 
whether or not there was any tenden епі p: 
to be strengthened directly, indep etha was 
Such distributional artifacts. A т senden 
needed which would be relatively пае ri он 
the distribution of responses follow! of erro" 
yet which would reveal strengthening 
This method is next described. 


stent 


d, 


‘ 565 
3. Intertrial Correlational Analy 


mag USEC i. 

The product-moment correlation уз еп a 

measure the degree of association This meas у 
and second-trial first айегешег ла e 

Was chosen because the distribution © 


FRROR-REINF 


MEAN TRIAL 1 CENTRAL-RE: 


SENT 
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TABLE & 


ST SCORES AFTER CENTRAL AND PERIPHERAL 


= MES REPEATED KEY RESPONSES 
Group Y After-position ciui 
j Р 
= 
ласа — 8 - 1 
Central 
cripheral ios 3.33 3.30 3.80 E 
EU A 13 3-23 3.20 3.12 == 
Central 
rak T 
Шы ¥ Mm ыш ш >= 
E Е: Е 3.12 è .05 
cro & 1) — 2 
-entral 
ae 
_ Peripheral га 3-40 3-50 3.27 -— 
CHE 3 | 3.54 3-4 3.23 
Central 
ИЕ: po mo HE 7 
- q^ 3-03 д> КЫ > 
core of one were included. 


^I : 
n orde н re 
r to increase the otherwise sme 


insensit; 
¥ Sitive ا‎ 

to deviations from the normal bi- 
tion in the parent population 


Vari R 
| p distribu 
Aou 
he observed r, however, mi 


һе 

Cn pr ; 
i oduce А ; B 

Key respe Cd by the relationship bet 


fin, 
al y 
th 
Corr ee the use of partial corre 
Clations were 


and tri 
ria 2 
l 2 first after-errors after key 


onse û » А 
Junction n and first error on trial 1, 
hese Felten a similar relationship on trial 2. 
ships can be partialled out of the 


computed between 


all N, Ss with an error-difference s 
ase and first after-error on 
the analysis was only for 
Ss who repeated the key response. Partial corre- 
lations were derived from these three corre- 
lations, SO that the intratrial correlations were 
partialled ou ation, which be- 


t of the final correl 
came а pure measure of intertrial correlation. 
Table 7 shows t 


he results. 
AM the jntratrial corre 
although severa 


and between key respor 
trial 1 and 2. Again, 


ght have 
ween the 
in con- 


Jations were negative, 


lation. 
Пу significant. 


trial 
response, 


1 
] were not statistica 


AND PARTIAL CORRELATIONS FOR 


INTER- AND INTRATR 
IR- AND INTRATRIAL CORR ERI ада 
E — First Errors А ;p KEY RESPONSES 
Gr a 7 Trial т Trial 2 Partial 
25 N Intertrial 7 Intratrial 7 Intratrial 7 Intertrial r 
E - 
After 
2 = A6? —.18 .26^ 
After 6 85 31° mee ud —.12 
otal 9r —.o8 n zs 58 +07 
PN. 152 I 
EU 3 
After ^ ^ 
2 —.38b 23 ‚21 
After 6 02 -30° as i وو‎ M 
otal 128 od L6 — 21 .21^ 
C 220 .28 
sor 7 —.ої —.ot лт 
CRB 
Afte b TT a —.02 
r2 ET s 
After 6 85 “1, pe 3 —.25 28? 
otal 97 T со? —.25 20 
182 p 3 
ac 
Signi 
» oi&nificantly different from zero at the .os level. 
t the -o1 level. 


Si £ i 
Enificantly different from zero а 
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TABLE 8 
SS F First Errors 
3 RIAL CORRELATIONS FOR 
ш AFTER CENTRAL AND PERIPHERAL 
REPEATED KEY RESPONSES 


Central Peripheral 

Saip N r N f 
E 

After 2 55 .39% 30 .30 

After 6 53 —.03 44 —.97 

Total 108 .20b 74 .07 
EU 

After 2 59 .365 33 221 

After 6 79 E. 49 .31* 

Total 138 .28b 82 .25^ 
C (1-8) 

Total 143 .02 58 ‚285 
CRB 

After 2 55 «34> 3o .02 

After 6 53 +23 44 ‚405 

Total 108 .28h 74 .24^ 


° Significantly different from zero 


i i at .os level. 
Significantly different from zero 


at .or level. 


The negative correlations reflect the alternation 
tendency. In the control group, the intratrial 
g this tendency was negligi- 


The difference bet 
т for trial 1 and the 
significant at the 05 level, or better, for 
E-2, EU-2, EU-6, CRB-2 
the same significance y 
and EU-2, EU-6, CRB-2, and CRD-6. Since onl 
Ss who repeated were i i analy- 
sis, the differences were not produced by any 
artifacts due to differential] repetition, Reward, 
as in the E and EU Broups, or forcing, as in the 
CRB group, thus seems to increase the Operation 
of response biases of the kind shown in intratria] 
correlation. 


After the intratrial effects had been ruled out, 
three out of the four experimental subgroups 
still showed significant Partial rs, indicating 
“pure” intertrial associations between first errors, 
However, a negative r was obtained f 


1 : or E-6, cor- 
responding to its weak error-reinforcement, 


Thus, when this &roup was pooled with E-2 to 
get the total E, the corre] 


smaller than that for the C 
significantly so. It is difficul 
should be such a marked di 
second and sixth position 
Broup. It was pointed out 
nation tendency worked a 
forcement in this group, b 
explaining also. In the a 


ween the contro] intratrial 
т for the other groups was 


ation was actually 
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explanation for this effect, an bep i da n А 
terms of sampling variation cannot bc тес fa 
The control intertrial r and its partial ee 
both positive, but were not significantly ol 
than zero. The differences between the pet 
partial r and the other partial rs ا‎ 
for significance, using the /' 918007 кас 
method (6). None of the differences peo 
cant. Thus the partial correlation tech! hae 
failed to reveal any pure intertrial «есі its 
differed significantly from E to C SEXUS. the 
still true that the obtained values of , ‘larger 
experimental groups were significantly. roup: 
than zero, with the exception of the È { 5 Ше 
An interesting incidental finding is ae on 
intratrial rs were without exception st e first 
trial 2. It may be that nonreward of п e dd 
after-error tended to weaken the tendon from 
make that response to the stimuli HERD 
the reward of the preceding key Ho thd a 
Similar correlations were run between 5 ‚ү re 
third, and fourth errors. following ше KOF ige 
sponse. The correlations were generally 
nificant. aluate the 
A further attempt was made 10 суа certos 
intertrial correlation between first deu о 
in terms of the central or анны e 
the key response. Table 8 shows the Se na 
In general, the results were what a atl 
expected on the basis of the central-rest relation 
The control groups showed a higher Y sitions 
following key responses in E жем sig 
while the experimental groups showed n centra 
nificant results following kcy responses 1n 
positions, 


re 


cond, 


be 


4. “Mirroring” Analysis 1 
; observe 
An apparent “mirroring” effect. Ps ameti 
in the experimental Ss (E and EU v response 
This was the tendency for Ss whose koy 5 of the 
and first after-error were on opposite poem the 
center of the apparatus to respond > ition о 
Same distance from the central-rest De atk, a 
the two successive responses. For exam} the 1€ 
Who responded with a no. 10 light АТ right 
warded response has gone 4 units 10 1 sides О! 
ОЁ the resting position, If he alternated d to re 
the Succeeding response, he would cs to the 
‘pend approximately the same вава to th 
left, that is close to the no. 3 light (4 ur 
left of the central-rest position). the co 
Evidence for this tendency is given by ате m 
relations in Table 9. These correlations sa 2 
terms of the absolute distance from um cionshiP 
Test position, There was a significant re агай 
etween the key response and first ps no case 
n the experimental groups on trial "These А] 
Was a control correlation significant. 8 


seco 
Toring effects largely disappeared on the 
tria], 


i 
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This a is i 
Ыш е suggestive with regard to the 
following Qe significant experimental gradients 
sumably, аи responses, As а result, pre- 
cantly larger 1e mirroring tendency, a signifi- 
Ss responded proportion of such experimental 
side followi, to the periphery on the opposite 
Out of 31 st a rewarded key response on trial 1. 
responses ail Group C that had peripheral key 
sponse, dE alternated their succeeding re 
In Ше нн А A to the peripheral area. 
such res К and EU groups, 07 
This diene were to the a moste б RI 
=o for rd E ey significant (2 = 872. 
era] first-errpr )). he total number of periph- 
key responses Responses that followed peripheral 
ранае 5 Was also significantly higher for the 
al than for the control Ss (85 of 156, 


Compared wi 
with 22 ais. کے‎ 3 
1 df). 1 22 of 58; x? = 4.64, Р = 05, for 


These diffe 
the trial Fiore are important because when 
егу, the inter t-crror response was to the periph- 
be high unl i difference score expected would 
оп trial g хы there was again a mirroring effect 
On trial s. UNE the mirroring effect dissipated 
Control than de a slightly higher proportion of 
опе would Sporuna al peripheral first errors, 
xperiment; expect higher difference scores for 
terms of ee . Thus the response biasing in 
бітага еген ane worked against intertrial 
Broups TNR gradients in the experimental 
that eme point is of interest in indicating 
Production se biases do not necessarily favor the 
Slowing b gradients, or strong first errors, 
"he eae ns rewarded responses. 
cated by. 4i gth of the mirroring effect, as indi- 
Suggests ite. first-trial cor ‘lations in Table 9, 
Portant an possibility that it might be an im- 
= the Аура to what has been described 
S. The or ral-rest tendency in the experimental 
Was made AS contribution to this tendency 
alternateq Pe Ss whose central first errors were 
toring and ollowing central key responses. Mir- 
Ounded in central-rest factors are obviously con” 
In seine na central-central alternations. 
Central-rest to determine whether а genuine 
аза factor tendency remained when mirroring 
Volving only à eliminated, a check was made in- 
assumption » nonalternating first errors. On the 
епсу oremus an independent central-rest tend- 
Proportion ing reward, it was predicted that the 
Peripheral m nonalternating Ss shifting from 
Proportion ш would be greater than the 
is Men. E from central to peripheral. 
UO for у d lon was confirmed (y? = 2-74 
dj un a on trial 1; y? — 816, P = 9! for 
Group eee 2). The comparable checks for 
ү тее of the f RB failed to show the same effect, 
ite directi our proportions being in the oppo- 
and Meu and the fourth being VCTY slight 

t is ee n (x? < 1.00). 

us concluded that the pres 


ent study 


IN A SERIAL PERCEPTUAL-MOTOR TASK 15 


TABLE 9 
CORRELATIONS BETWEEN CENTRAL-REST SCORES 
or KEY RESPONSES AND ALTERNATING 
First ERRORS ("MiRRORING" EFFECT) 


First Trial 


| 
| 

Group, AA AN Combined 
N r N r N r 

E 78 -15 44 5 122 6* 

EU |o 4075 40 132% 164 m 

C 93 o9 47 — .09 141 .00 

СЕВ | тоо E 40 ‚12 149 -10 

Second Trial 

Group, AA NA Combined 
N r N r N r 

E 78 21 33 +16 iit 10" 

ZU 115 -03 32 -25 147 -98 

C o3 -15 44 — -24 137— -02 

CRB | 200 =0 23 = .05 123— .10 


the .os level. 


* Significant at 
the .or level. 


** Significant at 


evidence for the 


does provide independent 
central-rest tendency as а side-effect of reward. 
independent of the 


mirroring 
ctor is of course a 


t peripheral-perip 


Evidence for 
central-rest fa 
the significan 


scribed above. 
The various intratrial com pa 


the present section were based on 
tabulations available from the A 


note 3). 


150 provided—by 
heral shifts de- 


risons made in 


summary data 
DI (see foot- 


Be Combined Alternation and Gentral- 
Rest Analysis 
alyses were an attempt to further 

in terms of combinations 
. The first analysis con- 
h of the major groups 
h tend to reduce the error- 


difference scores and within categories where the 
toward increasing the error-difference 


and NN are again combined, 
on the same side on 


mbined since 
ides of the 


These an 
analyze the gradients 
of response-biasing effects. 
cerns the percentage 
within categories whic! 


e first error Was 
N and NA are cor 


different si 


there was а rela- 
rcentage of each 
This indicates 


major group 3 € 1 
om re observed differences 1n gradients did not 
upon this difference in percentage of the 
sponses were biased toward low 


For example, the percentage of 


“Table 10, 
in the pe 


difference scores. 
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TABLE то uu | 
P NTAGE OF Ss IN VARIOUS CENTRAL-PERIPHERAL AND ALTERNATION CATEGORIES "— 
ERCENTAGE OF > AARD HE 
| 7 Central Peripheral a: 
| = a - — M 
i t Same Different е7 
етир | 4 Зате E Total Sides | Sides ^" Total 
(АА, ММ) (AN, NA) | (AA, NN Д TAN Ni == 
Е 182 36.2 23.1 59.3 | 21.4 3 pw 
EU 220 40.0 22.7 62.7 23 1 Е, 
G 201 42.8 28.3 0.3 $ 5 id 
CRB 182 13.2 46.2 59.4 | 5 


the C group in the AA, NN, central category 
was larger than for either of the other groups, 
and this category should produce the smallest 
difference scores on the basis of the central-rest 
tendency and the alternation tendency per se. 
The first-error scores actually found for cach 
of the subgroups are shown in Table 11, There 
an interesting difference was reve 
the C and E groups. In the C grou 
eral errors were strongest, In the E groups, 
the central errors were strongest, following re- 
ward. To check the effect further, special control 
data composed of repetitions by chance, not re- 
warded, in the E groups, were tabulated, As 
shown in Table 11, these repetitions were also 
followed by stronger peripheral errors, 
Inspection of the distribution of central and 
peripheral first errors following central and ре- 
ripheral repeated key Tesponses suggests an in- 
terpretation of this result, On trial 


aled between 
р, the periph- 


1 the ratio 


105 
in proportions че аи at hei 
level (y? = 3.23; 3.84 required for 1 ti aires 
proportion of central first errors Wen id error- 
a natural bias towards lower intertrial © 
difference scores, as has been shown iare re 

The greater tendency for peripheral. KC) 


errors in 
sponses to be followed by central f à p the 
the control 55 is explainable on the ры sare 
relatively lower number of periphe P ently 
alternating relationships (such as yer aj p» ex 
produced by the mirroring tendency in inity 


perimental Ss), and the consequent Шу че to 
for more or less random first-error КЫ пия 
occur more frequently therefore in the 
areas, 

Table 11 also shows the £ tests for the $ 
combinations of first errors. PUTAS ie 
key responses, two of the four subgrou] 


alye 
Hans 5 ined analy 
were significant. Both the comb gnifi- 


various 
ига 


roached si 
А ter central key responses Н ани е е) the 
of central to peripheral first errors was greater - Since certain of the ts st aa aN are 
following a peripheral key response for the con- accepted level of significance, their 7 y reate! 
trol Ss, and was greater following a central key — given. even though they were slightly Fa for 
response for the experimental Ss. This difference than .05. Thus an analysis which contro 
TABLE rr > 
MEAN dnd DIFFERENCE Scores FOLLOWING CENTRAL AND PERIPHERAL KEY 
ESPONSES, ALso CATEGORIZED py A -TERNATION CLASSES aa 
Same Sides NE ia Di еб ; 
i ifferent Sides “ombined 
Group (AA & NN) (AN & NA) хе ае 
3 =, Sr Tota 
Cent. Periph. Total Cent. Periph. Тош | Cent. Periph. 1 
.90 
E 126 1.33 ү 29 m 2 2 
I à : " 5 76 3.3 .90 
EU Ыз ото r9 a5 LE озы | 2.06 rs en 
C я 1.73 1.45 1.63 5.89 5.38 5.73 3.39 2.87 3.63 
E (C) т 1.64 1.58 1.61 5-04 $.10 3.64 3.92 3.18 3.00 
EU (C) 1:70 — 1.69 1,66 5-95 s.s 381 3.33 2077 0 
t tests 
Group | | m — 
E vs. C 2.12* 47 m SS apr o m" 
У E E 1.83 62 
EU vs. C 1.08 1.84 2.]1** e as 74 جب‎ 
* E (C) and EU (C) refer to first errors after ch эко 
* Significant at .os level, 


** Significant at .от level. 


ance repetitions in the E and EU groups: 


ERROR-REINFORCEMENT IN 


Sy pena ol central-rest factors still re- 
cance, and in e at least approaching signifi- 
Nono if E mic cases reaching it. 
significant am, D qan showed а clearly 
Ра wes a following repeated peri- 
pei pêra ا‎ Hover, differences following 
mental grou ee were biased in the experi- 
effects, as ө e a function of the mirroring 
they pen eee out in the preceding section; 
test of the s ue regarded as constituting a fair 
spread hypothesis. 


IV. Discussion 

В 4 ы e on the basis of the 
and d to make a single simple 
cerning tl ef poles Шеше con 
all of d гау of error gradients after 
Moved = re artifacts have been ге- 
comparis хе анка] experimental-control 
though — were not significant, even 
theñiselvas , жарашы! gradients by 
repeater а were generally significant for 
were n s and the comparable controls 

Ot. 
Meus other hand, а great deal of 
Various evidence was obtained im the 
first нн analyses neni m 
or ШУ t aera 
analysis result kd У тё inec a 2 а: 
the о» ] is 5 were significant at beyo 
groups, р for the experimental sub- 
relation aiia out of the four partial gor: 
perita s were also significant for the ex- 
кзн, subgroups, in analyses that 
such as s the distributional аиа 
the 5n : 18 central-rest tendency. Two 9 
Bisnes direct experimental-control con 
alternati in the final analysis combining 
signific ion and central-rest factors Were 
орь М for the central rewarded re- 

ез. Moreover, it was shown that the 
жез failure of the experimental 
of е to occur following e 
weli Vom rewarded. responses з 
int nave been an artifact of the bias 
tł roduced by the “mirroring” effect. Of 
Pa iu experimental-control patet 
еза. 9115008; ошу the partial 7s failed 

some significant differences. Even 
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here, three of the four subgroup differ- 
ences were in the predicted direction. 
The present experiment provides two 
additional important findings. One is 
that reward produces, as а side-effect, 
stronger intratrial relationships than 
nonreward. The second is that the ex- 
perimental gradients seemed to depend 
on the repetition or near-repetition of 
the rewarded response. 
On the assumption that the control 
oduced sufficient positive evi- 
fy continued concern with 
the problem, an explanation of the 
Thorndike gradients in terms of both the 
and the repetition of key response 
factors was attempted. This hypothesis 
be called the serial response-re- 
sponse reinforcement hypothesis. 
According to the serial R-R reinforce- 
ment hypothesis, the repeated rewarded 
response plays а dual role in producing a 
gradient. It serves, first, to reinforce, 
through its reward (Thorndikian) action, 
the responses that follow. This takes ac- 
count of the presumed superiority of re- 
peated-rewarded over repeated-nonre- 
warded responses in producing gradients. 
Second, it serves as a constant element in 


the stimulus complex. This takes account 
of the apparent necessity for repetition of 
e rewarded response. Thus the S-R 

if Thorndike's terminology is 
een successive responses, 
as well as between 


and responses, as 


analyses pr 
dence to justi 


reward 


will 


ed, are есм : 
as stimuli, 


imuli 
e stl 


Thorndike a B 
This hypothesis differs from the origi- 


nal Thorndikian “spread” hypothesis and 
also from Tolman's (18) suggestion of 
reinforcement of blocks of responses in 
that it refers only to the after-gradient 
and that it assumes the dependence of 
this gradient upon the repetition, 01 
repetition, of the rewarded sapone З е 
term “serial” is appended to indicate 
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specificity of reference to that kind of 
sequential response situation. 

The present experiment affords some 
evidence which supports this hypothesis. 
The intratrial correlations were with- 
out exception higher following reward 
than following nonreward. There were 
stronger alternation tendencies follow- 
ing reward than were found in the con- 
trol group. Central-rest gradients were 
found only following reward. All these 
lines of evidence indicate that following 
repeated rewarded responses, the re- 
sponses of the $ become more predict- 
able. This is not simply a function of 
the fact that the response was repeated, 
since the key response was also repeated 
by the control S. 

The CRB group is especially interest- 
ing in this respect. Here the $ was forced 
to make a given response on both trials. 
The intratrial rs were especially high for 
this group; three of the four were sig- 
nificant at beyond the .о1 level, and the 
fourth at beyond the :05 level. Yet there 
was no evidence of a gradient. This in- 
dicates the necessity of the Thorndikian 
type of strengthening to produce a gradi- 
ent. Thus the CRB group gives further 
evidence that biasing of response se- 
quences is not always sufficient t 
duce error gradients. 

More direct support for the seria] R-R 
hypothesis, which was admittedly ad hoc 
in the present study, has come from two 
subsequent experiments (7, 10) in which 
direct experimental-control comparisons 
have produced significant differences, 

Certain incidental results of the pres- 
ent experiment have interesting implica- 


tions and offer suggestions for further 
research. 


о pro- 


The central-rest tendency consisted of a signifi- 
cant tendency for Ss to reduce the magnitude of 
their response (amplitude of stick excursion from 
central-rest position) immediately following re- 
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ward of a central response. Theoretically, ke 
might be interpreted as indicative of an не 
caution, or perhaps of increased relaxation, as а 
function of reward. йрй 

The mirroring effect consisted of a sigm еч 
tendency for Ss to match their rewarded ж б 
in terms of distance of movement from кен 
central-rest position, when the first RHENO 
was made on the alternate side from the an 
warded response, This effect was most pe 
on the first trial, Theoretically, it offers e uS 
teresting support for the automatic cllects а АП 
ward, since it can be explained in terms © ИЕ 
immediate repetition of а response denial be 
reward, if the response is now considered t€ ilie 
а movement of a certain magnitude from 
central-rest. position. — аЙ 

Practically, these effects may be of шер 
possible biasing factors in the operation of € 
plex controls. 

Finally, Group CRB is of interest ir іх 
Ss, while generally not offering significant В forc- 
ents, did show certain indications of таун 
ing” influences. ‘Theoretically, the questo. rê 
raised is the extent to which “forcing с 9 take 
sponses, as used in this control group, m re- 
on “reinforcing” properties. The problem T о 
lated to the old theoretical one of the dia ems 
"exercise." This kind of forcing gann a ji in- 
to offer certain advantages for the furthe 
vestigation of the problem. 


in that these 


V. SuMMARY 


e. А t the 
This study was designed to tes 


Thorndikian spread-of-effect hypothesi 
that errors are differentially strengthen 10 
аз а direct function of their ШИШ it 
rewarded responses. More specifically, 
was designed to determine whether a 
tistically significant gradients of ер re 
Strength would still be found Sp 
moval of the guessing sequence and P di- 
ability-bias factors by control a a 
rectly comparable to the expen’ ard 
groups. Such direct experimental-co! 
comparisons have not hitherto | g 
made in the investigations purport®? p. 
show that gradients can occur in dd 

sence of the usual reward. Й 


vere 
à ach © 
Four main groups of зоо 55 сас! oii 
aM cn ; 
Used. Ss performed individually ratu? 
а 


: а б а 
modified complex coordinator арр 


(11). 
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e experiment groups. 
неа ak, - a exponat group Ss 
Ерык у 3 ге о prearranged rewards, 
made in the » F ae particular зеро 
коралла n inn positions. The second 
lu ness E Broup included a control 
voli ui. шш on the test trial. The 
рв aec group consisted of 300 
An Шап). e i the first two trials. 
group for ae 74 Ss were added to this 
ФП, КОПТ of the analyses. The ѕес- 
were indivi dah consisted. of Ss who 
Жай. e i ШШ matched with the 
tions сеа Ss. In the posi- 
warded E ретш Ss were re- 
sine оне. кеге forced to make the 
i5 уйы одр This group was designed 
Shut dise Г, вана check of the re- 
right ме аА tors, The Ss were told that 
бейопа! dic. were not involved since 
the БЫ S checks were being made on 
Th р: ratus. 

Were FE eer after-error gradients 
Ward. No da: the two groups given re- 
the ас i gradient were found in 
were found Fi No fore-gradients 
Were бон зе experimental gradients 

о depend upon the repeti- 


tion 

, 0 E ا‎ 

ê r near-repetition, of the rewarded 
„°5ропве, 


mg reli а, gradients follow- 
trol gradi п were significant, and con- 
Dut eein following nonrewarded 
there dm ed responses did not occur; but 
tween е i significant differences be- 
ents, n demain and control gradi- 
Significant ios. however, a number of 
Strengths of differences between the 
ни of the first after-error responses. 
Controlled epar sap artifacts were 
nique wh y using a correlation, tech- 
changes hich is relatively insensitive to 
results » in distribution of errors. These 
the Sid rig to agree with the results of 
ie a 5B of variance of the gradient. 

e a eren correlations tended 10 
icant, and the control not sig- 


nificant, but the differences between ex- 
perimental and control correlations were 
not significant. 

Partial correlations were used to take 
out the intertrial correlations which 
might occur as а result of a concurrence 
of intratrial correlation between key re- 
sponses and first error on the two trials. 
This was important because the intratrial 
relationships between the key responses 
and first after-errors were increased fol- 
lowing reward or forcing, in nearly every 
case, compared to the observed relation- 
ship following repeated key responses in 
the control group. The partial correla- 
tions showed the same results as the gross 
intertrial correlations: three out of four 
significant partial rs in the experimental 
ignificant partial 7s in 


subgroups, no 51 
the control group, and no significant ex- 


perimental-control differences. 

Although the apparatus was designed 
to be free of pre-experimental response 
ne interesting biases appeared 
in the experiment. First, there was а 
tendency for Ss to respond to alternate 
sides of the apparatus. Second, experi- 
ental Ss tended to respond closer to the 
center of the apparatus following reward 
of a central response. Third, those Ss who 

alternate sides of the ap- 
e key response and the first 


after-error tended to respond about the 
same distance from center on the two 
successive responses. This “mirroring” 
effect was most apparent on the first trial. 
АП these response biases were greater 
following reward. 

Refined control analyses were done, 
controlling for both alternation and cen- 
tral-rest tendencies. Some of these com- 
risons of first after-errors reached sig- 
and others approached it. Sig- 
nces were found only fol- 
d central key responses. 
tendency worked against 


biases, sor 


m 


responded to 
paratus on th 


pa 
nificance, 
nificant differe 
lowing repeate 
The mirroring 
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gradients in the experimental groups fol- 
lowing repeated peripheral key responses. 
This illustrates the important fact that 
response biases do not always work for 
the production of gradients. 

It was concluded that no single simple 
and conclusive positive statement could 
be made concerning the reality of the 
residual experimental gradients after re- 
moval of the intratrial response biases. 
Nevertheless, there was sufficient positive 
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evidence to justify continued concern 
with the problem. This evidence E 
gested a hypothesis, called the E 
response-response reinforcement piu 
esis, which postulates a dual role "en | : 
rewarded response. According to this T 
pothesis, it serves both as a consta 


element in the stimulus situation, 


thus 
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I. INTRODUCTION 


HE Officer E | 
T licer Education Research Labo- 
Trainin т Air Force Personnel and 
. x To ы 
itself for Research Center, has concerned 
evaluati some time with the problems of 

апо n а: 
Air Uni n of the effects of instruction 1n 
Work, E o In connection with this 
asked ucational Testing Service was 
training study the desired outcomes of 
Shoe in the Command and Staff 
former] В part of the Air University, 
re known as the Field Officer 

a 
005 to d nd to develop prototype meth- 
3 determi ‚ 
tives ar 2 mine how well these objec 

5 are beine achi Tos 
scribes a eing achieved. This report de- 
totype = attempt to develop such pro- 

1 А 
маз dir ethods, Although the research 
ёш ea uie toward evaluation of the 
um a 
ieee at the Command and Staff 
fen 
This r 
s. resear = 
part by MAC d was supported in whole or P 
tract No United States Air Force under Con- 
cation Roe 833, monitored by Officer Edu- 
and Train arch Laboratory, Air Force personnel 
Force pa ning Research Center, Maxwell Air 
jn ase, Alabama. 
Dri " : 
Sources eee was ‘conceived by the Human Re- 
Command А Institute when it was under the 
“ation has of Major General Carroll. This organi- 
Search Б become the Officer Education Re- 
ersonnel boratory and a unit of the Air Force 
Air Resea and Training Research Laboratory, 
Samuel Me and Development Command. Dr- 
ald у. Mai Goodman and his successor, рг. Don- 
hey and со» have acted as contract monitors; 
Numerous heir staff. have been very helpful O" 
wen cond occasions, The project could not have 
of Сао ed without the active collaboration 
'ommand Walker and Colonel Adams, successive 
"SC, i of the Command and Staff School, 
Planning ss University, and their Curriculum 
g Board. Lieutenant Colonel Sheeks ап 


A 


School and not toward assessment of in- 
dividual officers, the discussion has dis- 
tinct implications for the development of 
instruments to evaluate individual per- 
formance. 

The problems of assessment in areas 
demanding а high level of performance 
present a challenge to those interested in 
techniques of measurement. There 
ar need for instruments which will 


plex skills as the ability 
forma- 


new 
is a cle 
measure such com 
to organize discrete pieces of in 
tion, to discover the problems implicit in 
to anticipate events which 
cause o[ such problems, and 
to arrive at decisions based on a large 
of considerations. These and 
115 are continually demanded of 


ative officers in key positions. 
1 of functioning, tests of in- 


a situation, 
may arise be 


number 
other ski 
administr 

At this leve 


ant Colonel Wall, members of this Board. 
s our liaison with CSS. Colonel 
; of General Courses, acted as 
veen ACSC and ETS for the first phase 
j г. T. F. Staton, Director of Edu- 
ACSC, has been particu- 
of the study- 

a number of ETS 
e project at 
ties. During 


Lieuten 
served € 


Assistance in 
course 


‘Acknowledgment 
who 


Harold Gulliksen, 
г. Charles 


various Ways 
pr. Mollenkop£ aided in the 
in-basket problems. Mis: 
rovided technical help 
analysis data and Mrs. Marjor E s 
i provided assistance 11 scoring. De 
ge served as а consultant to the project 
Ne 
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ent O 
«eg Gallagher Р 
x of the 


[rving Lom 


2 NORMAN FREDERIKSEN, D. R. SAUNDERS, AND BARBARA WAND 


tellectual ability bear a lower relation to 
performance than they do to performance 
on tasks of a less complex nature, partly 
because selection on the basis of intelli- 
gence has already taken place, and partly 
because administrative responsibilities 
appear to demand additional skills. 


Several earlier attempts to devise situational 
tests which would evaluate performance at this 
level have been reported in the literature. For 

: instance, use has been made in Great Britain of 
such tests in selecting men for responsible posts 
in the Civil Service, for selecting industrial exec- 
utives (2, 3), and by the War Office Selection 
Board for evaluating officer potential (4). In 
Great Britain, however, the tendency has been to 
use group problem-solving techniques more fre- 
quently than written tests, In Australia, Lafitte 
(6, p. 107) reports the development of a written 
device which, in its conception, is not unlike the 
test described in this monograph. 

Considerable imagination and versatility have 
been demonstrated in the work on these new 
techniques. At the same time, the proponents (7) 
of these innovations have frequently expressed 
their scepticism of the value of objective, scora- 
ble, psychological tests. Thus, it is often the case 
that in attempting to solve the problems of eval- 
uation of performance in high-level jobs the 
test-maker is considered to be facing the choice 
of constructing objective, reliably scored, but 
relatively insensitive instruments, or developing 
a more sensitive measure which resists attempts 
to use it reliably and objectively, 


The instrument which is described in 
this report is the result of an attempt to 
devise a sensitive measure which may at 
the same time be objectively and reliabl 
scored, and proceeds from a faith that 
progress toward both goals of sensit 
and of objectivity may be made in 
operation. 

This instrument, which has been called 
the In-Basket Test, is a situational test 
presented in written form and group ad- 
ministered. The briefing on the nature of 
the problems and the presentation of the 
problems are carried out in such a way 
that the information available to the can- 
didate is the same for all candidates. The 
test allows a great freedom in response. 
The problems are presented in such a 


ivity 
one 


Way that it is up to the candidate first to 
discover the problem and only then to 
organize an attack. Although the In- 
Basket Test was designed to represent the 
situation faced by the Field Officer in the 
Air Force, material suitable to other 
areas of experience may readily be 
adapted to this form. | - 
It is hoped that the description of = 
steps taken in developing this instrumen 
and of the problems encountered along 
the way, as well as the recommendations 
for further improvement of the test, Шш 
be helpful to those who are working 0! 
instruments of this kind. Р ied 
The first phase of the research involve : 
a careful study of the curriculum and z 
the objectives of the course in order ts 
determine quite specifically what aP a 
of their work the officers were expected z 
perform more effectively as а Lage 
quence of the training.? Students pa P 
Command and Staff School ses Pr 
mostly of the rank of major and Pre 
tenant colonel and have been ag" 
selected for training to fit them 
greater administrative responsibilities e 
field grade officers. This training inclut Я 
courses with such titles as Orgena 
Management, Personnel, Intellige ain 
Operations, and Logistics. The por 
source of information about the obj 4 
tives of the instruction was a series 0 
interviews with instructors and oye 
officials. The instructors were ann 
state what on-the-job activities nore 
Would expect graduates to handle гис 
expertly аза consequence of the pa 
tion. An attempt was made to Бе d 
instructors to avoid generalities anc. 
describe observable behaviors 
would indicate whether or not a 5t" 


had attained the desired objective. 


vitaes. under 
2 n i t 

* This phase of the study was carried n n) 
the direction of Dr, Warren G. Findley * 
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4 
Л Altosetl = 
gether 25 е 
i S more than 500 such state- cedures which are consequences of the introd 
oduc- 


ments wer 
E ere c cte i 
ced re collected. In order to put tion of new research developments. 
агре number avi i | 
: 8 ber of behavioral descr 
Ws 4 a d descrip- Change: 1 ET 
E dem А hanges the organizational patte: 
ааа sired outcomes into а тоге without hesitation when it ош ко pied 
ө, sysiem, the next step was to Я 
Ssify the statements into categories = a cg apod not oniy. fon ы о ы 
£ bing, but also for tactical support a 
t low 


which w 
wc m 5 
| uld correspond to psychologi- level, for delivering napalm bombs, and for other 


cally М 
те; "нуз ; : 
У Meaningful functions. unconventional uses. 


Twely 
Ve catesories mak 
itegories make up the system These statements are alike in demand- 


ing flexibility, adaptability, and willing- 


Of classifica: 
assific: ; 
ds лш which was evolved. In 
2 Teen 3 x A 
marily re categories behaviors are pri- ness to introduce change. 
div b ium d 
Bovis r- б удин. that is, they аге be- Here are some additional examples: 
vhich could be exhibited by a 
Can comprehend the effect on AF activities of 
involy 
behaviors that are primarily in- 
{ 


person а 
` nds all a Noa i H : 
lone. The other six categories possible political-economic events such as the 
тни closing of highways and railways into Berlin. 
Тое; that is : А 
ds ; that is, they involve a rel: - id 
ship with other d 3 Se relation Anticipates and makes projections of his plans 
dividual c. r people. Four of the in- many months in advance. 
primar +t р were selected as the 
mënt y focus of the evaluation instru- prepares alternative operational plans, based on 
to b s different contingencies. 
TI € developed. 
пе cate "S ч rad- 
tegories may best be defined by "These statements assert that the grad 
uate of the Command and Staff School is 


St giving s 
Biving some examples of statements ў А д 
relatively likely to show foresight, that 
uture situa- 


fir 


and u 
le à 
to fies the term that has been chosen. NN f 
gory sent the statements in that cate- „Ма асири Фа feel at à м 
; tions. This involves anticipating possible 
as well as probable consequences and pro- 


Bel 

OW are 

obiectis. we some selected statements of `... 4 S 

vës which fall into one of the viding for contingencies. 1 
Here are some statements which fall 


Caen: 
legories: 

into still another category: 
ns to achieve a stated objec- 
able weapons first 


Cares 

шу f Е 

Y follows established supply procedure [pn selecting weapo 
and second from 


Under 
tive, considers the various avail 


И no " 
D Tmal circumstances. 
[z 1 i k 
ant Only with ‚н > from the viewpoint of efficiency, 
M 1 the civilian personnel officer іп the viewpoint of economy. 
advising concerning a partic- 


ütters du: 
5 of civilian employment. 
made, includes cost as an 


In deciding upon or 
ular decision to be 


Refr: 


ains fy 
то А > 
m making inappropriate requests of 


Units 

бп ап A; 

т " Air Force base, important factor. 

he: 
Se sta : я ы 
1 ate of y Statements assert that the gradu- plans for making appropriate use of various 
Пе Command and Staff School is weather conditions in combat operations. 

assert that the grad- 


These statements 
е Command and Staff School is 


luate data effectively. This 


t as to what data to 
d what to exclude 


ike] 

Y to ail: E E 
e p буй make efficient use of routines, 
e of z actions appropriate to the 
Nels lis assignment, and using chan- 


an e 3 = ) 
P d “SOP (Standard Operating will inv c 
include as pertine 


uate of th 
likely to ё0@ 
olve judgmen 
nt an 


9ceq 
ui 
i ed to advantage. 
a . . 
Эре o bel some examples of another as irrelevant to a solution. 
ne All four of these categories of behavior 
are individual behaviors. The other two 


«айцу pl 
ans for and makes changes in PT?" 


4 


categories of individual behaviors are 
knowledge and effective guidance of the 
decision-making function of the unit. 
Since the interactive behaviors were 
expected to require relatively unwieldy 
evaluation methods, priority was given to 
developing measures of the individual 
types of behavior. Also, some of the cur- 
riculum divisions, such as Communica- 
tions and Electronics, and Judge Advo- 
cate General, were thought not to justify 
immediate or extensive evaluation efforts, 
in view of the small proportion of time 
devoted to them in the instruction. Thus, 
four curriculum divisions—namely, Man- 
agement, Personnel, Operations, and 
Logistics—and four major types of in- 
dividual outcomes—namely, efficient use 
of routines, flexibility, foresight, and ef- 
fective evaluation of data—were selected 
to receive the main emphasis in the de- 
velopment of evaluation methods, 


II. GENERAL DESCRIPTION OF THE 
IN-BaskrT TEST 


Purposes of the Evaluation Instruments 

A measurement device was planned 
which would provide a separate score for 
each of the four selected functional cate- 
gories of behavior, and which would sup- 
port curriculum development by yielding 
information pertinent to the related cur- 
riculum divisions. Thus, the In-Basket 
Test resulted from an attempt to develop 
measuring instruments which would per- 
mit an evaluation of the extent to which 
students profited from the aspects of the 
instruction which aimed at improving 
their ability to use Standard Operating 
Procedure (SOP), increasing their flexi- 
bility, improving their foresight, and in- 
creasing their ability to evaluate data 
effectively. The intent was not merely to 
discover if the students had mastered 
textbook knowledge about flexibility, for 
instance, but rather to find out whether 
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or not the graduates of the course Бе 
hibited in their own behavior on the job 
the characteristics were being 
sought. From one point of view, a Сї 
terion measure was being developed as 2 
means for evaluating the effectiveness of 
Air Force administrative officers. 


which 


Description of the In-Basket Materials 


Р 4 le- 
The materials which have been € 


veloped are called collectively the "T 
Basket Test. A large amount of the ш 
work of an administrator centers aroun“ 
the contents of his in-basket. The IT 
Basket Test consists in putting @ candi- 
date into a realistic situation which л 
upon him to deal appropriately w 
such material as an Air Force office! 
might find in his in-basket. 

The form of the test which was © 1 
out at Maxwell Air Force Base geile 
eight hours of testing, during which eac 


А К M ;‚ roles 
candidate was required to play fout hà 
eriod th 
ind- 
tion 


5 tried 


in succession. In one two-hour p | 
candidate played the role of a Comm: 
ing Officer of a hypothetical Compos! м 
Wing, in another the Wing's Direct?" ds 
Materiel (D/M), and in other per? 


+ actor 
Director of Personnel (D/P) and pont 
of Operations (D/O). In each of Ur 

cont! 


roles he was given an in-basket а 
ing incoming letters, memoranda: * re 
studies, letters prepared for his sign? ma 
by subordinates, and other similar о 
terial. He was given suitable fonds a 
which to write answers, and his direct se 
were to go to work as though he ™ 
actually on the job. „єй? 
А соттоп, and often valid, alae on 
to situational tests is that the only d de 
able response to a situation is, afai! 
pends.” Examinees feel that it 15 ion 
to be presented with a complex situ? de 
in four lines and then be expect€ ^ yas 
scribe the best action, An attemP! ihe 
made to overcome this difficulty ш 


колаш M 


EE MTS 
y ~ 


THE IN-BASKET TEST К 
ә 


einn — Test by presenting 
make d nt MOTE of information tO 
unreasonable to say that the 
bit dea a answer is “It depends.” This 
able ае кү хауз. First, a consider- 
about dn pue background information 
Was ed е арас composite wing 
prior to rs ed for study by the candidates 
ie test " ume when they actually took 
TM м р ds information included such 
wing Я 4 slaitement of the mission of the 
Me. rige history of the wing, an or- 
кеңе. of a a table showing the 
allies of А ле wing, a roster of the key 
cation of оз Wing, maps showing the lo- 
maps indi ine City Air porce Base, and 
buildin ү the landing strips and 
tion e 2 the Base. Second, the situa- 
supposedty | arranged that an adjutant 
the indi 4 had studied the contents of 
files a i E and had added from the 
the as ODE documents to explain 
Needed nt : E memoranda and to provide 
ing a Pris m information. In mak- 
never iced: in a real-life situation one 
this ‘Sven the relevant information. In 
ably um сше text situation was prob- 
in real greatly different from a situation 
т} ife. 
oe iin of 
Was Sane Î f of the In-Basket Test 
tion witl plished through close m en 
involved ; sic of the Air Force, an¢ 
Staff athe att letters, memoranda, e 
actually = ey were like those whic 1 
Of an Air Елм be found in the in-basket 
Work is q orce officer. This phase of the 
ater wo in greater detail in а 
ion. 
When the candidat neared to t 
€ test, he | ў е appes À ea 
pportun; had already been given * 
terials у to study the background ma- 
- His instructions were roughly 25 


the problems 


ake 


role of Direc- 


Tod 
a 
tor ур Y JOU are asked to take the 1 
posite Wing. 


of р, 
*rsonnel of the 71st Com 


The previous Director of Personnel, Lt. Col. 
Hart, was killed in an auto accident, and you 
have been assigned to take his place. A manila 
envelope on your desk contains the materials 
which have collected in Hart's in-basket together 
with additional material placed there by the Ad- 
jutant for your guidance. Your job is to read 
your mail and take appropriate action as though 
you are actually on the job. Write the appropri- 
ate notes, memos, letters, or directives. Take as 
much action as you can with the information 


available to you. You are limited in that no more 
1 during the next few 


information can be obtaincd 
hours and you can communicate only in writing. 
In his Director of Personnel in-basket, 
the candidate found such items as (а) а 
letter for the policy file from Col. Good- 
fellow, the Wing Commander, which 
states that it will continue to be the 
policy of the wing to pay constant atten- 
tion to the problem of morale; (b) а 
letter from the Personnel Officer regard- 
ing the conduct of Airman Third Class 
Joe Doakes, who is a personnel problem, 
asking that Doakes be given a talking-to; 
(c) a note from Col. Goodfellow request- 
ing that an appropriate policy statement 
ardship and bad conduct 


with respect to h 
discharges be drafted; this note is backed 
up by а memorandum from the Legal 


Officer describing how punishment had 
been escaped by an airman through the 
hardship discharge, and another docu- 
ment showing that the former Director 
of Personnel has concurred in the Legal 
Officer's recommendation; and (d) a 
memorandum from the Civilian Person- 
nel Officer regarding @ visit by a delega- 
n of civilian employees who have a 
ance. These аге only a few of the 
d in the in-basket. 

There is no one-to-one relationship be- 


problems and documents pre- 
didate. Some of the doc- 


uments are inc ely as sta tements 
of policy—as additional background in- 
formation. Sometimes а problem is rep- 
resented by 2 combination of two or 
three documents which are not physically 


tio 
griev 
problems foun 


tween 


sented to the can 
included mer! 
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together. In other instances the real prob- 
lem may not be the one stated by the 
writer of a memorandum, but rather one 
which is implicit in what is stated. 


ПІ. How THE In-BAsker TEST 
Was DEVELOPED 


Development of the Setting 


Logically, the first step was to develop 
some concept of a basic situation or situ- 
ations in which the problems would be 
set. After some of the problems had been 
substantially prepared, it was decided to 
place all of the problems in the setting of 
an imaginary "ist Composite Wing.” 
Since all four roles are placed in the same 
imaginary wing, a minimum of confusion 
was created for the subjects who are re- 
quired to play each role in turn; also, an 
opportunity was created for the exami- 
nees to carry over information learned in 
one role to subsequent roles, By placing 
the imaginary wing in Maine, an oppor- 
unity was created to use problems de- 
pendent on adverse weather conditions 
and, by making the wing a “composite” 
wing, opportunity was created to use 
problems involving several types of 
craft. 

The actual process of developing the 
situation went hand in hand with the 
development of specific problems that 
would fit the situation, until a consider- 
able body of factual detail about the 715 
Composite Wing was built up. Forms 
were developed, tables of Strength and 
organization were drawn up, statements 
of mission and policy and records of 


air- 


per- 
formance were created, all in order to 
provide the background information 


needed to set and solve the problems. It 
is believed that the background informa- 
tion provided was broad enough to make 
it reasonable for candidates to take ac- 
tion on a majority of the problems. 


Use of Essays in Constructing Problems 


The first approach that was systematt- 
cally used to create problems for the S 
Basket Test involved study of a series О 
essays written by the students in the CSS 
soon after they arrived at the school. Stu- 
dents were asked to describe some prob- 
lem in the Air Force—how it arose, ш 
factors bearing on its solution, wag 
been done to solve it, and what теа ы. 
to be done to complete its solution. 1 E 
essay was written as if to brief a ана 
to the job when there was no Өрү 
for personal briefing. Approximately mi 
of the materials developed for the Ps 
1953 tryout of the in-basket proce? Ц 
were derived from ideas in these essays: 


з , —9* 
Use of Interviews in Constructing 
Problems 


= wm o tap 
It was also considered desirable t Air 


i 
some source of problems remote from ов? 
University in order to eliminate any т Ў 
sible bias toward, ог away from, Р E 
lems familiar to students in the pe to 
Accordingly, arrangements were mAN IEN 
visit McGuire Air Force Base, ad- 
Jersey, for the purpose of developint в 
ditional problems for the test. Inte? 
were held with about a dozen и Air 
officers in the Headquarters of кез di- 
Defense Wing and in the bep fc 
rectly under this wing located = the 
Guire Air Force Base. The — Jems 
interview was always to isolate am t the 
which looked as if they would met о. 
requirements of the in-basket ge^ each 
An attempt was made to block о acti 
problem in sufficient detail so ж Р uld 
writing of the necessary materia n 
proceed. This included a diset um: 
the general nature of each memor? would 
letter, or other document that i 
be necessary to prepare. 


ЧУ — b; CIA. 
ы? NOME 
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Steps in Problem Preparation 


do game steps typically were ap- 

нена н фе ing a problem for the In- 

oe est rom materials obtained 
sources discussed above. 


ia Y 
eek ү was selected in the devel- 
tation wot a problem at which its presen- 
teadi ald require а minimum amount of 

ing. 
E ae pad whether or not the prob- 
in the ety related to one of the four roles 
D/M, DIO Compose Wing (Commander, 

e boc D/P). 

problem, Pu rx mined for presenting the 
limes involv xi role. This method some- 
Several me ed dividing the problem up into 
would be ai рак so that the problem 
tionships Каз only by noticing the rela- 
other cases pp the memoranda. Or 1n 
poor soltitic ' жеө handled by preparing a 
the е з юр BE sentation along with 
passed forw; 2 since correspondence that d 
orward without recommendations is 


properly re 
lion," y returned as "incomplete staff ac- 


Lr 


sary to present the 


- Materi; 
Crials that were neces 
le 


orc ч 
А еле written, An effort was mac 
riefly an " iat these materials were stated 
ing of ihe clearly, so that the understand- 
much on Iu would not depend too 
tions were ending comprehension. Indica- 
Seen by pin that the materials had been 
them Бего eryone who should have seen 
(unless Nes they got into the in-basket 
test the stu йо of the problem was to 
ordination). recognition of incomplete 
5. Othe Л 
that might ШЕ documents м є 
ture of th be needed to complete the PIC 
he problem or to provide informa- 


tion 

that чу i 

solution. would be necded for the intended 
n. 


ere written 


iewed 


> Th 
© pr 
Y H 
Problem materials were then TeV 
ni- 


familiar with Air Force term‏ و 
rect and ensure that the language was cor-‏ 
terials Куше een and that the ma-‏ 
Normal co not unintentionally deviate from‏ 

7. Severa] mang and staff procedures. 
tryout جو‎ were asked, as ап informal 
take on : indicate the action they would 
that most sufi oblem; this helped to ensure 
апа that subjects would not be sidetracked 
Would h the differences in performance 
€ of the intended kind, in response 


the : 
intended problem. 


Development of the Procedure for Ad- 
ministration of the Test 


In the main tryout of the test in July 
1953, the following steps were taken in 
introducing the testing situation and in 
attempting to build up motivation: 


1, The proctor reads a statement out- 
lining the ground rules of the test situa- 
tion and “explaining” how the subject 
had gotten into it. For example, the fol- 
lowing instructions were for the role of 
D/P (which was not the first role played). 


a. Today you are asked to take the role of the 
Director of Personnel of the 71st Composite 
Wing, Pine City AFB, Maine. The previous 
D/P, Lt. Col. Charles Z. Hart, was killed 
four days ago in an auto accident, and you 
have been assigned to Pine City AFB to take 
his place. Today's date is Tuesday, 6 July 
1954- The manila envelope on your desk 
contains the materials which have collected 
in Hart's in-basket, plus additional material 
placed there by the Adjutant for your 
guidance. . . 


b. Your job is similar to the previous ones. 


You are merely to read your mail and take 
appropriate action as though you were ac- 
tually on the job. Write the appropriate 
notes, memos, letters, OF directives. Take as 
much action as you can with the informa- 
tion which is available to you. As before, 
you are limited in that (1) no more infor- 
mation can be obtained during the next two 
hours, and (2) you can communicate only in 


writing. 


c, Please write legibly. place the appropriate 


file number on each sheet, and sign your 
own name. ] have additional copies of Form 


p Wg if you need bim 
i vou have finish t 
a itis back into the envelope, including 
the materials you have written. place a 
aper clip 9n all the papers you have 
in order to separate them from 

гоп have not written On. Leave any 
DWS outside of your enve- 


а, place all of the 


ee 

were asked to use Form 71W3 
s. This is a form, de- 
hich is similar tO 


e Air Force, but 
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lope. Write your name and course code 
number on the outside of the envelope. 

е. As you know, the findings of this test will 
have no bearing on your grades, but may be 
very helpful in the development of evalua- 
tion methods. Therefore, you are requested 
to cooperate by doing your best. 

f. Go ahead. 


Development of the Scoring Procedure 


Once the problems and a procedure 
for their administration had been de- 
veloped, the major remaining problem 
was the development of an adequate sys- 
tem for scoring the test on the various 


categories of functional behavior. 

Having administered the series of problems in 
answers were drawn for 
ed for scoring purposes. 
which will be re- 
, and III. Sample I 
of one student ran- 


after. constructing Sampl 
drawn from the cases rem 
ing Sample II, by rando 
cers from each cubicle, 1 
ble to stratify the sampl 
cubicle assignment in order to eliminat 
possible effects of communication betw 
jects during the course of the te. 
that there was some commun 
the cubicles, but it is belie 
largely confined to 
administration. 

The responses of the student 
in Sample I were sorted into ca 
ing the various types of respon 
For example, concurring in 
might be one type of resp 
concurrence might be indicated in а Breat variety 
of ways); refusing to concur might be another 
type; and referring the problem to higher author- 
ity might be a third type of response. Such cate- 
gories were further combined in an effort to 
arrive at a relatively small number of types of 
response that were clearly distinct from one 
another and easily definable, Between five and 
fifteen types of response were identified for each 
of the various problems, based on a study of the 
responses in Sample I. 

In setting up these types of res 
necessary to ignore a great deal o 
and to abstract from each response 
of the response which were relevant 
tional category of behavior (e.g., 


aining after construct- 
mly selecting three offi- 
ї was considered desira- 
es on the basis of their 
е апу 
een sub- 
sting. It is known 
ication in some of 
ved that this was 
procedural matters of the test 


3 to each problem 
tegories represent. 
se to the problem. 
а recommendation 
onse (although the 


ponse it was 
£ information 
those aspects 
to the func- 
foresight or 


o 

flexibility) which the problem was soppa а 
measure. Even striking differences from 5! з 
to student in style or tone of the peas 
were ignored in setting up the types отар. Be: 
This abstracting of a limited aspect о feat öf 
havior necessarily meant loss of a АО aoei 
information but presumably had the E variable 
of making the scores reflect only the ¥ 
being measured. | sand fiaustudy 

The types-of-response lists, obtained sing them 
of Sample I responses, were tested by t Br was 
to classify the responses from Sample ө segnem. 
found necessary, in order to make bedi the 
work for Sample II, to redefine or pa Оп 
concepts of some of the types of a from 
the whole, the types of response ИЕЫ 
the first sample were found to be adeg ү АЙ 

Оп the basis of the study of ani o be un 
Tesponses, some problems were found a in the 
suitable because of too great a uniformt in types 
answers or because of too great a Sh 50 
of response, Such problems were clim D ar in- 
far as further scoring is concerned. m the pro» 
stances it was found advisable to score 


be- 
ry О 
i ‘tional category е 
lem for a different functional c To measure: 


havior than it was originally designe meu final 
Following the study of Samples Seat for each 
lists of types of response were prepa 10 


submitted 


iy Force 
Air 
Ai in the 


remaining problem. ‘These WEE 
two panels of expert judges in ш nts 
One panel consisted of twelve siure ane! on 
Air War College (AWC). The other PAS Com- 
sisted of twelve staff members Гол. which C 
mand and Staff College (AC 3 50) (elude 
is a part). Each panel was chosen fo ies the 
officers particularly qualified to Jus V les o 
lem materials in cach of the four ! 
In-Basket Test. E 
Each subpanel of three ipe sk 10 
consider the problems іп (ће inks ng values Е 
appropriate role апа to assign mp | pre р 
€ach type of response listed for ANC wor $i 
being scored. The subpanels from d ШП АС ° 
independently from the subpanels team. 
SC, but each subpanel worked 2 snider d e f 
The judges were first asked to comank-oF Ic! һе 
basket as a whole, and to assign a cored: the 
importance to the problems being blem bY ing 
level of priority given to each pro EA 


asked E 


[4 
t for tP 


y 


ining the 5^ ne 
А H 2 f 
Judges was later used in determine С m 
Weight assigned for failing to ! “the rà nds 
Problem. The agreement between bl 


of two subpanels of judges was е to € 

perfect. The judges were then asker а 

cach scorable problem separately, to cach в 

Scoring values on a five-point scale the scale 

of response, such that cach point on 

used at least once for each problem: 
The amount of agreement vp 3 

panels of judges on the value to a е, ram 

the types of response was quite vat 


==% 
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TABLE ı 
CORRELATIONS BETWEEN SCORES ASSIGNED BY Two PANELS OF EXPERTS AND SCORING 
RT SIDES FINALLY ASSIGNED TO TYPES OF RESPONSE FOR EACH 1х-ВАЅКЕТ PROBLEM 
C y Correlations H 8 
| orrelations Betweenza Сш» Correlations Correlations 
Probl | Between Scores Seite cr the Between Scores Between Scores 
em Assigned by Two | chores of Both Assigned by AWC Assigned by ACSC 
| Panels of Experts | panels and Final and Final Scoring and Final Scoring 
(AWC and ACSC) Se rins Weights Weights Weights 
Gey ооб а = = 
C213 | ae | n — -— 
(aug 130 62 = — 
ee 00 | 84 = = 
496 | 38 | .96 TS — 
C525 | 28 | -02 = = 
ese .82 -92 a 
С? .66 .86 E 
7 | 7 .00 =, = 
Co83 Mi .81 ж == 
Mo8; е9 | 80 = = 
Mar 77 | p» == = 
Maj Е 178 | = = 
M325 E? 06 — e 
M Soe .66 55 = = 
55 M 88 — 
М916 | 29 82 = pe 
Тодт a ii оо 1.00 
Posg А -— = 
id 35 95 .62 17 
р 7 20 т | — 
›1395 =з | a 
P247 оо = ^3 59 
P406 me 88 г 20 
5461 959 pa -94 = se 
526 COND = 
Põo7 43 ы ‚70° 27 
„OI 
0116 _ 87 +28 
290 .07 8 os = 
E 72 (0 | = T 
500b 05 pS | Б .58 
Sool) =з | ax a = 
568 +22 E = 
0795 .40 | > = — 
786 -54 : = .91 
~ 946 166 93 —.28 9 
" —.28 | 2 i 
elow <35: separate correlations 


In 
With «OSes w а falls b 
th the Aw There the agreement between the Air Force panels 

9 final sc and ACSC panels are reported. 


Oring weights assigned. 


fro 
the 
Corp, SOOM 7 anel, and 
Tak tions Ө poor. Occasionally even negative AWC tances the tw 
lem ® 1 prese Observed. The first column of other INS" eement; 
5 the correlation for cach prob- — were in PO? MA 
bs lly found -— н 
t Е with 
am ide “st-makers, of course, had their substantia l the problems ™ 
оша ж about what the scoring weights ° ly two 0 о great 75 to 
a Was | In the case of some problems ? reemen е droppe 
su x blem 
thr bstantial agreement between p decisions d 


се 
8toups—the AC & SC panel, the 
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final scoring values were generally ar- 
rived at by the following process: 


1. A careful examination was made of the 
weights suggested by the Air Force panels, in 
order to determine which of the original func- 
tional classes of behavior was apparently being 
given greatest weight in determining their judg- 
ments. These determinations were compared with 
the original notion of the functional class of 
behavior for which the problem had been written. 
A decision was then made as to the functional 
class to which the final scoring of the problem 
would be assigned. By this time, a number of 
problems had become assigned to another of the 
curriculum objectives, namely, Guidance of Dc- 
cision-Making. 

2, The various types of response were ordered 
by the score developers in terms of their knowl- 
edge of the functional category of behavior de- 
termined in step 1. In doing so, they attempted 
to avoid being influenced by the detailed rank- 
ings made by the Air Force experts. This pro- 
cedure was designed to ensure that the final 
scoring weights would reflect as nearly as possi- 
ble a single functional category of behavior, but 
would at the same time correlate as highly as 


possible with the judgments of the Air Force 
panels of judges. 


An indication of the success of this 
procedure is provided in Table 1, in 
which are presented the actual correla- 
tions between the composite (over-all 
average) Air Force judgments and the 
final scoring weights. (In cases where the 
agreement between the Air Force panels 
as shown in column one of Table 1 falls 
below .35, separate correlations with the 
AWC and AC & SC panels are reported.) 

Scoring a paper, then, consisted in 
reading a response to a problem, compar- 
ing the response with the listed types of 
response for that problem, deciding to 
which type the particular response be- 
longed, and then assigning the numerical 
value which corresponded to that type. 

Sample III of the responses selected 
from the CSS data were not looked at 
until a final scoring summary sheet for 
that problem had been prepared and the 
scoring manual had been written, The 
results obtained in the scoring and anal- 


ysis of Sample III are presented in the 
next section of this report. 


IV. RrsuLTS OF THE COMMAND AND 
Starr Scuoor TRYOUT 


i " 1. were tried 
The "in-basket" materials were trie 

i ыб m 

out in July 1953 by administering si 
3 C nc 


to the entire Class 53-B of the Commat 
and Staff School. The administration be 
quired four two-hour sessions, which 
were distributed at intervals of anon 
three days through the second and thirt 
weeks of the course. Each student in p 
course played in turn the roles of el 
mander (CO), Director of аген d 
(D/M), Director of Personnel (D/P), ai 
Director of Operation (D/O) of the 7! 

Composite Wing. The purpose 0° 
July 1953 tryout of the materials wa de- 
provide data which could be used to ta- 
termine some of the operational and pei 
tistical characteristics of the In-Bas' 


s 10 


will be discussed under four та] 
ings: Scoring Reliability, Over 
ability, Validity, and Attitudes T 
the Test. 


Scoring Reliability 


Reliability of scoring may be 
by comparing the scores assignet gon 
scorer with scores assigned by a 5 
scorer, and is expressed here m 
the product-moment correlatio 
cient between the scores assigne. 
scorers of the same set of response" 


terms 


n coe 
1 by t^? 


fi- 


ti 
" А ¿pected 
The scoring reliability may be exp 


a function of the amount and quality ©, ining 
that the scorers have had. Арртор! ence 15 
would ordinarily involve direct ехре in di 
scoring a large number of answers “iy otf, 
cussing the scoring of these answers ехре деле, 
more experienced scorers. Relevant ket Te 
with the job situation which the In- fl 
is designed to simulate would be very 
learning to score the test. 
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TABLE 2 


ELATIONS BETWEEN SCORERS A, B, AND С 
FOR SINGLE DIRECTOR OF MATER EL 
PROBLEMS AND FOR D/M 
'TorAL SCORE 


Scorers Scorers Scorers 
| A&B B&C A&C 
| N=50 N=20 N=20 
| „QI 78 8o 
| 8s — » 
58 87 .85 
«84 7171 ES 
bo .76 .00 
63 .66 47 
Py i .48 51 
рум On .8o .89 .94 
E otalScore| — .83 69 .68 


^ Scores е 
cores not available for scorer С. 


„н of scoring reliability data 
is [ог ie outed: The first set of data 
whom ae independent scorers all of 
least iori - regarded as having had at 
Two of s ЖЕЦУ previous experience. 
Marily т ча Scorers (A and B) were prr 
of Red npe for the development 
and dies cae procedures and manual 
en ice ы the best scorers 
the ec on С had some contact with 
аз With eid situation portrayed, as well 
ems in I enge administrative prob- 
intimate] бада work, but had not been 
Ment of á associated with the develop- 
Telations $ scorning procedures. The cor- 
first po reported in Table 2 are for the 
ems roii of Sample III D/M prob- 
Cases for scorers A and В and the first 20 
Scorer C. 
Scoring mae set of data pertaining 10 
Scorers, p ps is for two independent 
Tectly hes D. Scorer D worked di- 
anual wj nm final form of the scoring 
нер hout much special training 
th ae benefit of prior experience 
а anger situation. These data 
b w reliably the In-Basket Test 
Ostly ны by someone эе 
inimum the printed instructions with 
of special training. Table 3 


With 
Indic 


may 
m 


presents the resulting correlations for 
each problem, for the total score for each 
role, and also for the grand total score. 
Even under these conditions scoring re- 
liability is reasonably satisfactory. It is 
clear from these data, considered as a 
whole, that the In-Basket Test can be 
scored with a reasonably high degree of 
reliability. 


Over-all Reliability 

The problem of determining the over- 
all reliability of the In-Basket Test is 
somewhat unusual. The rationale of the 
test calls for the measurement of a series 
of distinct, conceptually different dimen- 
sions—namely, the functional categories 
of behavior identified in the study of 
curriculum | objectives. Therefore, one 
aspect of reliability which was of interest 
was the degree to which problems de- 
signed to measure the same dimension 
of behavior did, in fact, measure the same 
thing. At the same time, the structure of 
the test provides four distinct roles, which 
are the separately timed portions of the 
total test. This permitted a test of the re- 


liability of the total score of the In-Basket 
Test under the assumption that the four 
lent forms of the 


roles are four equiva 

test. Thus, a two-way breakdown of the 
problems is possible; and in designing 
the testing battery an effort was made to 
provide problems that could be sorted 
into the cells in such a two-way break- 
down. It should be noted here that all 
values indicating over-all reliability are 
based on the scores assigned by one 


scorer, scorer D. 


vidence bearing on the initial 
es that underlay the preparation of the 
ional cate- 


f the 


5 to 
t or subtest, however; 


12 NORMAN FREDERIKSEN, D. К. SAUNDERS, AND BARBARA WAND 


TABLE 3 


senum ee Р 5 SOLE 
CORRELATIONS BETWEEN Scorers B AND D ron SINGLE IN-BASKET конш R 
TOTALS, AND GRAND TOTAL FOR COMMAND AND STAFF SCHOOL SAMPLE 


Commanding Officer Problems 


Director of Materiel Problems 


N=112 
Problem Correlation 
Coso .92 
C213 -75 
C229 .87 
C446 -89 
C496 81 
С525 .83 
Cóso -93 
C705 .81 
C747 -76 
Co83 .88 
Total Score on CO 
Problems .91 


Director of Personnel Problems 


Х= по " А 
Problem Correlation 
Mo87 -71 
M215 „81 
M278 81 
M325 .65 
М534 ‚84 
Mss2 .62 
Mo16 .69 
Модт .90 


Total Score on D/M 5 
Problems “74 


5 а : = T3, ams 
Director of Operations Problem 


N=102 
Problem Correlation 
Poss .8 
Pro; 73 
Р247 -79 
P406 28 
P461 .8o 
P526 .8o 
P607 .70 
Total Score on D/P 
Problems .8о 


№ = 106 r 
Problem : Correlation 

O116 a 
O290 4 

Озот s 
Osoo "a6 
O568 Se 
0705 qe 
0786 de 
0946 -83 


Total Score on D/O 86 
Problems сы 


Total of All Problems r= -90 


generally positive correlations wo 
in order to show that the items a 
in the sense that they all tend to measure the 
same ability, Such homogeneity is a sufficient 
condition for high reliability. 
. Intercorrelations of items int 
particular functional categorie: 
Tables 4, 5, 6, 7, and 8. It is e 
not a high degree of homo 
problems. In the cases of Categories II, ПІ, Iv, 
and V (the four categories originally established 
as the objectives for measurement), the correla- 
tions are predominantly Positive; in the case of 
the problems placed in Category VI, Guidance 
of Decision-Making, even this cannot be said. 
Category III (Flexibility) evidently shows the 
highest amount of homogeneity, and Category 
VI (Guidance of Decision-Making), the lowest 
amount. Category VI was not originally included 
among the functional categories to be represented 
in the test; the large representation of this cate- 
gory in the test resulted from the reclassification 
of problems because of the Opinions of the Air 
Force committees. All problems in this category 


uld be required 
te homogeneous 


ended to measure 
5 are presented in 
vident that there js 
geneity among the 


e 
Were originally written to represent other = 
tional categories, ugh 
None of the categories shows pond of 
homogeneity to warrant cota ll 
à separate reliability coefficient. 1 reli- 
à coefficient were computed for soos 1 
ability of the Category III total pop 
is guessed that it would be equal to û 
-40 Or .50. 
It will be noticed that in some of the count 
of intercorrelations, one or two probem s. 4 f 
for most of the negative values. In T E 
example, problem O290 accounts for bo plem 


zs rol 
negative correlations, and in Table 8 PU est 
d inary 4 


e tables 


P461 is consistently negative, As in oN an! 
construction procedures, the НЕМЕЕ be P 
hence the reliability of the score СОП h тас 
Proved by climinating these items whic the 


о 

bu: cent f 
from the total test. As it is, seven eri Jeve e 
correlations reach the one per сеї 
significance, 


INTERCORREL. 


TABLE 4 


TIONS OF 


THE IN-BASKET TEST 


PROBLEMS PLACED IN 


TABLE 5 


FUNCTIONAL CaTEGOR 


y III 


INTERCORRELATIONS OF PROBLEMS PLACED IN 


SCTIONAL CATEGORY II 
(Use or ROUTINES) (FLEXIBILITY) 

Problem | C446 Моб Póo7 Огоо Problem | Созо M215 Pro; P247 0568 
C446 | 10 од —os Coso or 31** оз or 
Mo16 10 o8 10 M215 or 04 12 42** 
Рбоу 09 o8 —os P107 31** o4 —o6 оў 
Озоо —os io: —os P247 o3 12 —06 її 

———— PRU 0568 от 47* оў її 
** Significant at the .от level. 
TABLE 6 
INTERCORRELATIONS OF PROBLEMS PLACED IN FUNCTIONAL CATEGORY IV 
"— (FORESIGHT) 
—.... Problem Cu; Мз Ми Р 006 Озо — Ores 
С?47 —o2 o6 18 її —06 W 
Мз —o2 oo 17 е 1$ =16' 
Most o6 oo o2 —o8 —08 13 
526 8 17 o2 o9 95 us 
0116 т 3:'* 08 o9 28% — —os 
Озоо —o6 15 —o8 os 28** ae 
= Ores 17 —13 13 12 —03 3 
** Qr. 
Significant at the .or level. 
TABLE 7 
" А / 
INTERCORRELATIONS OFZPROBLEMS PLACED IN FUNCTIONAL CATEGORY V 
c (EVALUATION ОЕ DATA) ы а 
c — — € = ۴ 
Problem | Сбзо Мо87 М278 P406 Озот 0786 Ооз 
Cóso = —17 o3 99 cl 
Mos, i % 2588 —о2 21° 1] B 
M278 - —3o** o5 DA Be 11 
P406 —17 —o2 o5 98 or o6 
Зот °з 22* —14 eB 04 
786 p 17 13 27 95 РА 
946 = —ot 28** 11 
94 Signifi 3 
EL vacant at the .os level. 
Mhcant at the .or level. 
TABLE 8 si 
=p IN FUNCTIONAL CATEGORY 
INTERCORRELATIONS OF PROBLEMS PLACED Т CARING) 
=. (GuIpANCE OF DECISION = ہے‎ == 
Prob = Pos8 P461 
s И M325 M552 

lem C213 C229 C496 C525 C705 C983 3 

Сатз s 14 97 1 os 20 

6229 o9 22* o5 E o8 —23* = —о1ї =11 

(496 o9 25* —02 e d 13 —03 E II —o6 
Us 22* 25% —19 I —її 5 6 —oo 

C225 5 o6 o9 -Fi DL. 

705 o5 —о2 —19 оо mar 14 01 
Cog; —95 —o2 —10 o6 —25 29 02 —o3 
M3 14 8 09 6d * 93 —ot 
M 5 =, о 13 02 —25 II 
p.532 97 —23*  —o3 ег E 20 o3 35 “60 
poss 13 —06 — o6 o5 6 14 02 аў —o9 

46 1o E II о! = —о3 РЕГ 

І 03 or s00 от ee مج‎ 

" E uj —1I — o6 


leni 
Meant at the .os level. 
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The lack of evidence of a high degree of homo- 
geneity should not be surprising in view of ind 
subjective nature of the categories, An empirica 
approach to the problem of classification into 
functional categories may prove to be more valid 
than the judgmental approach which was used. 

The next problem to consider involves 

the qaestion of whether or not the In- 
Basket Test as a whole may provide a 
meaningful total score. Evidence on this 
point is provided by the intercorrelations 
of the total scores for the four roles, and 
the correlations of these with the grand 
total score. This information is presented 
in Table g. The numbers in Table 9 vary 
widely, with the correlations between role 
total scores ranging from .06 to 45. The 
three. smallest correlations all involve 
the relations between total score for D/P 
and the other scores; none of th 
correlations is statistically significant, 
The other three correlations, which do 
not involve the D [Р score, are 
cant beyond the :05 level, usi 
tailed test of significance, TI 
from the D /P score, there 
some homogeneity in 


ese three 


all signifi- 
ng a one- 
1us, aside 


appears to be 
the 


materials 
covered by the different roles. (Whether 
the D/P score is a reliable measure of 


something else, or is merely an u 
measure of the same function measured 
by the other three roles, cannot be an- 
swered by the data in Table 9.) 
'The information contained 
9 provides a basis for estimati 
ability of the total In- 
works out to be .5o for the eight-hour 
battery, and .42 for the six-hour battery 
obtained by omitting the D/P problems 
from the total test. The former estimate 
of .50 is probably too high because the 
assumption of equivalent parts of the 
test is not satisfied (8). 
These values for reliabilit 
are of definite statistical sig 
not of sufficient magnitude 
interpretation of an indivi 


nreliable 


in Table 
ng the reli- 
Basket Test, This 


у, while they 
nificance, are 
to justify the 
dual’s scores 


TABLE 9 
] SC. 5S AND 
INTERCORRELATIONS OF THE ROLE асе 
CORRELATIONS ОЕ ROLE CORES WITI 
TOTAL SCORE on тї Ix-Baskrt т 
COMMAND AND STA CHOOL SAMPLE 


Grand” 


| CO D/M D/P D/O тоа 

Cd : [ 68 
CO | 24 14 19 es 
D/M | 24 15 45 55 
D/P | 14 15 o» 07 
D/O | 19 45 об Rl 

Grand | " 

Total» | 68 69 5o 67 E 
^N-92. . :rand 
^ Correlations of Role Scores with the Gt 


5 ih since 
Total Score are, of cour puriously ш the 
the Grand Total Score is based on the 9 s" value 
four Role Scores. The expected "chanc 
of these correlations is about .50. 
m the 
"T. ng 
on the In-Basket Test. Applying it 


Spearman-Brown prophecy Гой о 
appears that approximately 24 a rder 
testing time would be required m F » 
to achieve a reliability of the order о the 
or more with material like oe 
present In-Basket Test, However, !t is 
be remembered that the present en 
the first such instrument constructed, f 
the present tryout is а pretest ol ш 
experimental form. It seems Paine’ 
that higher reliability could be ae? , de 
by selection of the best problems, a ac 
velopment of better problems to Ar y 
those found to be unsatisfactory, соп 
improvement of the scoring cates 
and methods, 


irst 


justify comparing groups, for 
ing correlations between Ке irpos 
ous criteria, etc. The original PYF, 
for which the In-Basket Test "i " 
were all of this type, and it may ue ili 
be concluded that, insofar as im of the 
concerned, the original objectiv 
test can be met using the mater“ 
available in this research. 


. H i е 
Another important object» 


ND с ааа 
'—_— ج‎ 
c — ms, 


oe Бл fhe differentiation of several 
in the ав outcomes of instruction 
be obtained p» hoped at scores could 
make it possil на testing that would 
Success of d н а determine the relative 
these differ be instruetion in achieving 
any Ero dinem desired outcomes. Is there 
WOU en m in the present data that 
Mon a, Sang ati meaningful 
is, it аы pe e differentiated? If there 
the a нн ауа amen for the failure of 
clearly refle PE of behavior to be 
In order 2 - a ше марай findings: 
of one or Ane iake possible the discovery 
complete tal үн Өнер of behavior, a 
all the Жы ыш. ат among 
all four eat m problems included in 
Was фе. as computed. This table 
clusters of d in an effort to isolate 
Which all "9 least. five problems for 
tively large Б татан: меге rela- 
the sense den consistent in direction, in 
rat if a problem correlated 


negati 
atively wi + 
tiv ly with one of a number of post 


el 
correla : 
rel Y correlated problems it must cor- 


ate А м 
на ami. with all of them. 
clusters meeting this requirement 
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TABLE 10 
CLUSTER 1 
Problem | М215 Os68 O500 0786 P461 
M215 — 
0568 42 — 
Osoo 27 19 — 
0786 24 15 17 — 
P461 —16 —14 —20 —12 — 
TABLE rr 
CLUSTER 2 
Problem | M2:s Osoo 0568 C496 P461 
Mass = 
500 гї — 
0568 42 10 — 
C496 22 24 14 — 
P461 —16 —20 —14 -II — 
TABLE 12 
CLUSTER 3 
Problem | Osoo М534 O116 P461 Озот 
ын зке кшн ы жр „шысы 
Озоо T 
М534 15 Бы 
O116 28 31 b: 
P461 —40 —23 =1 IS 
Озот 27 14 12 —18 — 
TABLE 13 
CLUSTER 4 


Wer 

in Dore four of them are presented 

cluster, ii 10, 11, 12, and 13. The [ourth 

Variables e be noted, consists of six 
5. Several general observations 


may b 
e m: : 
н made concerning these tables. 
Шыр 


told, involve, all 


» the 
€ four clusters presented 
most 


i : 
опе iat different variables, which is al 
clusion 1 the individual problems available for 
We va A clusters, If clusters of four instead 
Maining ie were sought, many of the F€ 
m i ion ON could be brought into clusters. 
dene of the Bre: to justify the inference that 
тер се of ТЕНЕ ividual problems do possess some 
беа, TI bility over and above mere scoring 
r ‘ability с, ne obtained estimates ОЁ over-all 
*Broupin ‘ould thus be improved by appropriate 
tb. Second of problems. 
nc КУЫ 2710741 observation is that not all of 
mà тб ce ате positively intercorrelated xA 
ticula lusters. In 
» e dae P461, which occurs in three 
Ons wii ers, has consistently negative correla- 
1 the other problems in all three 


0 8 
Ё the problems in their € 


КЕ Ps26 Рбоу C213 Co83 Mss2 Pos8 
| ucc e == == = 


Р526 | — 

P607 17 = 

C213 | 25 28 = 

C983 | 24 p ous 

Mss2| 14 20 13 y >= 

Pos8 17 o9 10 14 її — 

clusters. А related observation is that the prob- 
of functional Category п 


js measures 


lems scored à 
nes) tend 


Use of Routi 
correlations wi 


Jems scored for 1 ٤ 
behavior. In particular, negative correlations are 


а between roblems scored for Class II 
(use ot pe nei) and Class Ш (Flexibility). 
Both of these observations suggest that the de- 
sired outcomes of instruction in the Command 
and Staff School are not merely nonhomogene 
ous, but are to some extent contradictory and 


to have more negative 
blems than do prob- 
ional classes of 
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conflicting in the manner of their interrelation 
within the student population. The possibility 
must seriously be considered that achievement 
towards one objective will simultaneously repre- 
sent movement away from some other objective. 
The achievement of a proper balance between 
objectives thus becomes an even more difficult 
problem. | 
A third general observation based on Tables 
10 through 13 is that it might be possible, by 
some method such as factor analysis, to obtain 
an empirical set of classes of functional behavior 
which would describe the domain of the prob- 
lems better than the a priori system of classifica- 
tion, Such a factor analysis would need to be 
applied to the complete table of intercorrelations 
and would be more likely to provide useful re- 
sults if an even larger number of problems were 
available for simultaneous analysis. In view of 
the generally small magnitude of the problem 
intercorrelations, it would be appropriate to con- 
duct the factor analysis according to some rigor- 
ous method, such as the Lawley Maximum Like- 
lihood Method, to ensure that the bounds of 
statistical significance were not being exceeded 


by the number and nature of the factors ex- 
tracted, 


It is of possible interest to offer an in- 
terpretation of some of the clusters rep- 
resented in Tables 10 through 13. Each 
cluster is likely to represent a group of 
problems that would turn out to have 
loadings on the same factor if a f 
analysis were performed. 


Let us first consider Table 10, АП of the prob- 
lems in this cluster seem to imply a desire on 
the part of the officer to make judgments suited 
to the merits of the individual cases, without 
regard to standard procedures which 
not indicate the same conclusion, Thus, in prob- 
lem M215, both the best and poorest answers in- 
volve acceptance of the endorsement to an in- 
spection report, but the best answer also indicates 
the intention of conducting a special inspection 
of the ammunition storage facilities, This prob- 
lem best represents the cluster, in view of its 
relatively high correlations with the others, In 
problem 0568, both good and poor answers in- 
volve following a training regulation, The good 
answer wants to establish the “j 


А intent” of the 
regulation, and then to provide an SOP “iiiter 


preting” it as it applies to the present situation, 
whereas the poor answer merely calls for “strict” 
compliance. In problem Osoo, both good and 
poor answers call for examination of a three-hour 
turn-around time, which is too long. Here, the 
good answer calls for an initial meeting to dis- 
cuss the problem; whereas the poor answer 


actor 


may or may 


plunges immediately into a detailed qur Ө! 
the problem, which may be unnecessary. nip dis- 
lem P461, which indicates that nda eed 
charges have been given to airmen in cases Ww s 
a bad conduct discharge would have been hes 
appropriate, decisions that move S maii 
preparation of a policy statement on the н tins 
generally are regarded as better than е Es 
ing toward consideration of the individua this 
and we observe, accordingly, that «ОДО the 
problem have a negative correlation Wit) 
rest of the cluster. 

The cluster presented in Table т бан, 
problems іп common with the one just Hits 
and can be interpreted along the same р ш 
logical lines. Most of the problems in the ened 
represented by Tables 11 and 12 were avior. 
to measure the Flexibility category of id 
They may perhaps be thought of as терте ibility 
a more homogencous, “purified” зе! of flex 
items. e solu- 

Let us now consider Table 12. All of ier 
tions correlating with this cluster seem 10 pleins 
a willingness to take definite action оп i prob- 
that clearly call for definite action, even о" ааста" 
lems that clearly call for prolonged oer renee? 
lion. Thus in problem M534, which Das terms 
sents this cluster, the scoring is entirely e the 
of the amount of concrete action s 
direction of grounding possibly defect! and i$ 
aircraft. Problem O116 is very simil: „action 
scored in terms of the amount of ponani С 
facilitating ап investigation of the fpu of th 
Ci19 carburetors, In problem 0500, О ШУ a 
two best answers is to set up imme огай 
maintenance training program, while Mis a any 
answer is to do nothing. In problem К оп 
definite decision regarding a policy st oq a 
hardship and bad conduct discharges e deci- 
poorer score than an answer delaying 


+. given © 
at all is 8 blem 


aO 


11 has four 


as : 
jn earns 


roblem*? 
ere orig" 
ght cate 


Which contains the intercorrclations of а st 
lems. All of the problems scem to eon 
@ working knowledge of the organitza ajni ir 
appropriate functions of various offices, 1 Я aking 
the office currently held by the subjec ake! jn 
the test. The exact nature of the aeron 
each problem varies. Thus, in problems д! А 
P607, the basic problem needs to Бе e night? 
ап appropriate person, at either lower in 
echelon. In problems C213 and C983’ 


more 
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for i 

смес to be gathered, and the recom- 
кетү po d basic problem need to be 
mu problem. P526, there is enough in- 
problem NI > Justify immediate concurrence. In 
ич | nd : — should be given, but per- 
Mast of Ae о s project should not be taken. 
Period de problems in this cluster came to be 
опи measiires of the “Guidance of De- 

Making" class of behavior. 


oe d evidence presented sup- 
Dui. wus de ts of the armchair analysis 
ancien that an improved basis of 
be бып i = ution) OB IEGINES could 
opment " ЗЕ would permit the devel- 
йк ae рашы homogeneous sub- 
uai E oam reliability [or individ- 
might етен Е urther improvement 
the scarin eei айне refinement of 
of iini A ү: из attached to the types 
seem to ыл he present test materials 
group — an adequate basis for 
studies тар зген and correlational 
lems A utilizing the group. of prob- 
for stad the experimental in-baskets 
tent Rc debe is some evidence of ‘con- 
Basket Tur 4 The reliability of the In- 
is неам = its present pretest form 
(those же for certain limited uses 
of өнд 3 comparisons of groups 
velopment i: size); and with further de- 
sufficiently | ле reliability might become 
in individ, righ to justify use of the test 
ual selection or placement. 
Validity 

iene validity may be thought of 
validit p dx The simplest concept 
dictive vei 5 predictive" validity; рте 
Correlatio, v: ity may be measured by "e 
Some ае between scores on the test anc 
€ test een of success in the activity 
asket Te Supposed to predict. The In- 
Orecast Sie is not primarily intended to 
2 неа It was intended rather " 

e Sse 7 of achievement which wou 
a in the evaluation of instruction. 
dateq can an achievement test be vali- 
It might be argued that an 


TABLE 14 


CORRELATIONS OF R 
ATIO) г ROLE TOTAL Score: 1 
GRAND Тоталі, SCORE WITH ACE Toral aR 
Score, AND CSS FINAL GRADE FOR 
CSS Samece 111% 


т АСЕ cs 
Variables Total E 
Score Grade 
Total Score 
CO Problems 224 di 
Total Score 
D/M Problems —.ої ло 
Total Score 
D/P Problems ‚12 12 
Total Score 
D/O Problems 25 o2 
Grand Total Score A 15 


a N=02. 
achievement test does not need to be vali- 
dated; insofar as the test directly repre- 
sents the skills it is intended to measure, 
the test is *self-validating." Probably no 
one would quarrel about such а defini- 
tion of validity for a test of (say) long 
division. In the present instance, how- 
ever, there might be more disagreement 
about whether or not the test does repre- 
sent the skills taught; the objectives of 
the instruction in the Command and 
Staff School are not quite as neatly de- 
fined by a set of test items as are the ob- 
iectives of an arithmetic course. 

In the long rum the validation ofa 
test depends upon building up a wealth 
of experience which is consistent with a 
ar definition of a test. We tend to 
accept à test as valid to the extent that 
over а period of time it is found to cor- 
relate positively with the measures which 
we feel the test should correlate with and 
t with things we feel it should not 
ith. Gulliksen (5) has called 
pt of test validity “intrinsic 


particul 


correlate W 
such a conce 
een In-Basket Test 
kinds of data have 
of these other vari- 


betw 
and four other 
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TABLE 15 i 
» E EYES axe EXE {кт TEST 
BREADTH OF EXPERIENCE AS RELATED TO ROLE TOTAL SCORE ON THE ІХ-ВАЅКЕТ E 
s = - & 
Narrow Experience 
Primary Air D/M | D/O DP " 
Borce ES | * ШЫ mela 2/5 Es | п Кш. Est 
E Mean SD Mean SD Mean 50 
4 2.14 4.05 
sonnel 16 14 1.14 "em 1:01 
e itration 19 19 2.16 9 2 ae 1.37 
Maint. & Supply | 8 8 —o.25 ч pr 5-42 
Operations 34 35 1.31 =. 3 3 d 
F=0.70 
F=2.05 Е =3.69 Hus m a 
Not significant +05>p>.o1 Not signitican 
іхрегіепсе 
Ргітагу Аіг — E 
Force Specialty co D/M 4 
Code n Est. Est. n Est. Est. n n 
Mean SD Mean SD 
Personnel 2 3.00 2.83 2 —1.50 о. 2 2 
Administration 8 0.88 2.80 4 4.50 6.32 5 5 
Maint. & Supply 8 1.00 6.87 8 4.00 3.21 7 5 
Operations 3 —2.оо 3.61 9 1.44 2.30 9 9 
e F —0.34 F =0.07 
Not significant Not significant Not significant = 


Not significant 


ables is a “criterion.” The correlational 
results are presented merely as a contri- 


bution to our understanding of wh 
and what is not me 


Basket Test. 

The four kinds of data are as follows: 
(а) scores on the American Council on 
Education Psychological Examination 
(the "ACE"); (b) CSS course grades; (с) 
statements about type of Air Force ex- 
perience; and (d) statements about 
breadth of Air Force experience, 

Table 14 presents the correlations of 
the first two kinds of data with the four 
role total scores and over-all total score 
on the In-Basket Test. The over-all total 
score correlates .25 with ACE total score, 
which is clearly significant from a Statisti- 
cal viewpoint. Such a correlation with a 
measure of mental ability is to be ex. 
pected in a test that measures any sort of 
intellectual function. 

Since the ACE total score and the CSS 
grades are correlated only to the extent 
of —.o2 in our sample, we may regard the 
CSS grade as being completely independ. 
ent of ACE score. The correlation of the 
in-basket total score with CSS grades was 


at is 
asured by the In- 


found to be .15. While this esate е 
on the borderline of statistical sE od 
cance when evaluated with a ptem 
lest against the null hypothesis, a, ol 
it does not represent a relationship be 
much practical importance. It Um 
determined from the available и = 
whether the low correlation post 
here is due to relatively low yo 
for the CSS grades (as well as for r tO 
present form of the In-Basket Test) dt 
lack of any strong relationship eae 
the two Measures. In view of the © that 
vations noted above which suggest may 
Some of the objectives of the goune 

prove to be diametrically appo 
other objectives of the course, wi 
haps should not expect a very hig that 
relation. It must be remembered also arly 
the In-Basket Test was administered ped 
in the course, not as a final semi yo 
Table 15 presents information ji data 
ing the third and fourth types ©! jided 
mentioned above, This table is а © 
into two parts, according to brea large 
administrative experience. The he toP 
group of students, represented at t e ha 
of the table, are considered to hav 


Nm 
cot- 
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ер tee administrative experi- 
Specialty рзи шен Pumary Air Force 
КЫЛЫ : Es e (PAFSC) at the time of 
ihefe a » the same general area as 
The а ве агеа of greatest experience. 
eag امد‎ of the group, at the 
vera e table, are considered to 
TRE "s бы in d broad experience, be- 
didnot ad UN at the time of testing 
as their I : inn the same general area 
ffo ao experience. Within these 
divided. PIES the cases have been 
Erentese нена to the stated area of 
Elem PAL ience, the areas having been 
possible wa 10 correspond as closely as 
"Tec oft е the roles of the In-Basket 
the mean he Г subgroup the table shows 
of the EL ы deviation for each 
The oie of the In-Basket Test. 

Using Ке Table 15 were analyzed 
values а чой variance, and the F 
Paring th en resulted are shown. Com- 
Sübgrou "i мачы variance within these 
Е ноч, E find that all but two of the 
of eight e due than unity, and one out 
05 level of 5 reaches significance at the 
Number of conndeng. In view of the 
Coes not wa isin made, even this result 
“should ean further consideration. 
Seneity of ишга that the great hetero- 
based no Variances within the groups 
Oretically of greatest experience the- 
Sver, the a ES the F test invalid. How- 
Scion in ем produced by this imper 
he data would be in the 

^ of producing apparent signifi- 
Sults ona it should not exist. Since the 
Mie TM not appear significant anyway» 
AM оо seems to be surely 
eni area of greatest administra- 
do шше has little if anything tO 
Test, „formance on the In-Basket 
Decus, i is an important conclusion, 
^. ^ for ii that there is no neces 
Construct ns other than face validity 

a special form of the In- 


dir 
lr ctio; 
Cance w 


just 
ti 
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Basket Test to examine officers in various 
areas of specialization. 

The other comparison permitted by 
the data of Table 15 is between those 
individuals having broad and those hav- 
ing narrow administrative experience. 
These comparisons were originally sug- 
gested by a close study of the results for 
the D/M problems, where it was found 
that those with broad experience had 
D/M total scores significantly better than 
those with narrow experience, at the .05 
level of confidence. As soon as the scores 
for the CO problems were available, and 
the data were subjected to the same kind 
of treatment, it was found that the officers 
with broad experience did better, and 
that this difference was significant at the 
.01 level of confidence. However, when 
the relative performance of these groups 


on the D/P and D/O problems was as- 


sessed, no significant differences were 


found. 

None of this infor 
question of the validity of the In-Basket 
Test. The findings do represent a be- 
elopment of our un- 
derstanding of the nature of the test. No 
conclusive evidence that the test is valid 
can be offered. In the final analysis, valid- 


ity is confidence in a test which is gen- 
borne out by numerous observa- 
test over a period of time. 


mation settles the 


ginning in the dev 


erally 
tions about the 


Attitudes Toward the In-Basket Test 


Following the ad 
-Basket Test, а m | 
ante to the CSS students. This mem- 
randum provided the students with ad- 
dit i ion concerning the pur- 
t in which they had 


ministration of the 
emorandum was 


osals for possible 


ht pro : 
De he new techniq e c ee 


s 


ications of t x 
in the memorandum. Theft. 
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TABLE 16 


Pi 7 FUTURE 
JUDGMENTS OF 111 CSS STUDENTS WITH REGARD TO EIGHT PROPOSALS кок Fu 
USE OF THE “1х-ВА5кЕТ” TEST MATERIALS 


Judgment €— 
| "Possibly " 
Proposed Use of Test^ Suitable Not . No 
F "Suitable" "Or Dount Suitable’ Comment 
ul Suit- 
ability” Нар. 
1. Basis for officer evaluation at command j 4 
and staff levels Д 22 то ^ 
2. Basis for selection of the superior officer " 5 
for special assignments А 28 9 38 9 
3. Basis for group comparison : 66 7 Ie 
4. Absolute standard of performance in com- 7 
mand and staff level jobs 6 $ 82 
5. Indicator of group strengths and weak- 6 
nesses 65 8 21 
6. Realistic work sample for preparing officer 2 
to face his new job 51 6 4 8 
7. Instructional materials 59 9 шы 
8. Further technique of final evaluation in 6 
school such as CSS 44 14 36 eT 
Note.—Each cell shows the percentage of the total group expressing a judgment. 
^ For exact statement of Proposed uses see page 24, 
ste RÊ 
i " А " ast indicate 
1. To provide a basis for uniform, impartial It is requested that each student inc 
evaluation of officer eff 


ectiveness at the command 
and staff level, 


2. To provide a basis for selecting superior 
officers for certain types of assignments, 

3. To provide a basis for comparing one group 
of officers with another group which may have 
had different job or educational experience, or 
both. 

4. To provide an absolute Standard of per- 
formance in command and staff leve] jobs. 

5. To provide information to a School such as 
CSS concerning the general Strengths and weak- 
nesses of a group of students, 

6. To provide students in a 
with a realistic sample of the 
meet in command and Staff j 

7. To provide instructiona 
most of the possible altern: 
advance. 

8. To provide a further t 
evaluation of individual stude 
as CSS. 


School such as Css 
work they will later 
obs of the Air Force. 
1 problems for Which 
atives are known in 


echnique of final 
nts in a school such 


The students were invited to ev 
the test materials in the light of th 
proposals, and to include suggesti 
the improvement of the materials 


saw fit. The questionnaire Was y 
as follows: 


aluate 
e eight 
ons for 
if they 
vorded 


answer to the following two Кеша уо! 
(a) Considering the test materials S dl point 
them, and answering from your p. for the 
of view, do you consider them ает на your 
purposes stated above in 2-c? (Inso ccific) : 
answer may be negative, please be SPE uggestions 
(b) Do you have any comments ыр purpose? 
for improving the test materials, for lo, please b 
Stated above in 2-c? (Again, if you ps that АР” 
specific. You may omit any ae test ma 
plies only to a single instance in t 
terials.) 


1 saw 


б» 
А mem 
Of the 500 or so copies of the гете 


randum that were distributed, M 
filled out and returned. The Жа ma 
the respondents chose to cog relat 
terials generally, rather than "opost 
their comments to any specific i in whe 
application as it was outlincc 


memorandum. ir 


ih 
: relate , 
However, 111 of the students did : 1 


г oP 
comments to specific proposals. TES лі 
ате represented in Table 16. With been ` qut 
of 111 the tendency seems to n iat techni g- 
prove of applications of the moon th on be 
to groups, but to object to its use a i wil 
tion of individuals. These comments, 


et 
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rem 5E : 
рки io the actual test materials 
proved test whi quie test idea or to some im- 
Of the een ee be developed. 
terials were ailea vig oe Be mu el 
to offer in Rie ui. but had no suggestions 
remaining 182 ut of any of the proposals. The 
pier wow: m ie the materials in a 
comments [3 p ing no attempt to relate their 
Sponses exhibi ле specific proposals. The re- 
varying fio ibited a great range of opinion, 
to К и profound skepticism of the test 
The T IG appro val | 
for а ges criticisms pointed to a need 
volved сане = briefing materials. Some in- 
tion p Cora or amplification of the informa- 
reference sce the briefing, such as more 
fuller БҮ: PEU on the policy of the wing. 
information шн and functional charts, more 
some аш predecessors’ duties, and 
terial prior ris 2 the handling of classified ma- 
Perhaps nis placing in the in-basket. 
of the tose ЕП тоге vital to the ultimate success 
should anu сы suggestions that the briefing 
of the У zn appropriate "set" on the part 
the Bids d Some felt that the importance of 
Stressed if fu [аш һауе been more heavily 
Others felt ill cooperation was to be obtained. 
Planation и a fuller and more frank ex- 
helped in ol | ıe purpose of the test would have 
A large s aining this cooperation. 
might be sed of students expressed what 
ing eei d a as hostility toward the test- 
artificial" res due to what they felt was an 
munication Smia on normal modes of com- 
İs an unimi Many suggested that memo writing 
and that бэл, tool of the good staff officer, 
Sonal conta is skill is exhibited, rather, in pet 
Some e with his staff. 
attitude dieu respondents suggested that this 
on Cres the test created by the restriction 
Y more а might be somewhat allayed 
ricfing ee preparation іп the initial 
. Tt was suggested tha the examiners 


display j 

limitations the briefing their awareness of these 

je to e students felt that specific ref- 

se dicate heres by which the student coul 
2 content of hypothetical phone calls, 


b meetin de 
© helpful gs, or informal conversations woul 


A] 
агре 
Problems BAe of students objected to the 
$ n the basis that they were "unrealis- 


" alu 
u 10u 
mon eal m the problems have been based 
Poo, 16 high pm. in the Air Force. Others felt 
Ше} aff Жш оп of problems exhibiting 
base or de 
Som e ава wh "m unrealistic itte “ 
Я Whole | 
Criticisms 
ms w " 
“рен the de were directed toward other 
eie o DU Mif ee from actual 
nity for h as failing to provide an op- 
Broup discussion and cooperative 


solution of the problems. Some felt that insuffi 
cient background material precluded any basis 
for wise action. Those who acted, they felt : would 
be making superficial judgments. Others were 
convinced that the short time allowance militated 
against considered judgments. 


V. RECOMMENDATIONS AND SUGGESTIONS 
FOR FURTHER RESEARCH 


The present In-Basket Test represents 
a first attempt to develop an instrument 
for use in evaluation of education at the 
very high level of complexity which exists 
in the Command and Staff School at Air 
University. It is a prototype of an in- 
strument which might become valuable 
not only for training evaluation but also 
for individual selection, placement, and 
guidance. Now that the pretest of this 
measuring device has been completed, 
recommendations can be made about the 
use of the In-Basket Test in its present 
or an equivalent form, and suggestions 
can be offered which have to do with 


further development of the test. 


Recommendations for Use of the Test in 


Its Present Form 


1. The In-Basket Test may justifiably be used 
in making comparisons between mean scores of 
groups of examinees. This is the use for which 
the test Was originally intended. Thus а group 
of CSS students tested at the beginning of the 
course might be compared with a group tested 
at the end of the course. Ora group of CSS 
graduates might be compared with a group of 
officers who are he graduates except 


similar to t ‹ 
hat they hav been students 1n CSS. If the 
кой to be red are reasonably large, at 
least some 


cores for roles and for 

jonal be used in such com- 
ыар а$ ге. Analysis of co- 
arison: 2 
variance i 


d method of analysis i 
suitable measures Ca 


be found to provide à 
basis for controlling 2 


bility factors in these com- 
arisotts. 
? 2. The 1n-Basket 


Th t 
instructional material. 
pne ie followed у 
in the em А rudents would s em to 

vel сане" i js is a US 
on een instructional device. Т M n sg be 
test which was not planned 1 
es 


of the part 5 
categories co! 
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warranted to the extent that instructors found 
the problems suitable for their objectives: TE 

3. The frequencies with which the P oae 
types of response occur in an administration o! 
the In-Basket Test may enable instructors to gain 
greater understanding of the results of their in- 
structional efforts as they apply to specific per- 
formances of students. For example, instructors 
might find that in view of the large proportion 
of students making one of the low-rated types of 
response, many students must have failed to 
gencralize some point in the instruction. On the 
other hand, they might be pleased to find that 
a new method of presentation had incrcased, 
from one class to the next, the proportion of 
students giving a “good” type of response to sev- 
eral similar problems. Such detailed analysis of 
performance on an examination provides maxi- 
mum guidance to the instructor who wishes to 
improve the quality of his teaching. 

4. The test might be used as a method for 
assessing a group of entering students in order to 
find out their general level of capability in the 
skills measured by the test. Thus the level or 
amount of instruction might be varied in ac- 
cordance with the ability of a particular entering 
class, or the students might be "sectioned" into 
two or three levels of ability in order that the 
level and amount of training would be more 
suited to their needs, 

5. The test should not, in its present form, be 
used as a basis for individual evaluation, either 


for selection, placement, guidance, or measure- 
ment of course achievement. 


Suggestions for Further Research 


The pretest form of the In-Basket Test 
represents a novel approach to a measure- 
ment problem, and little recorded ex. 
perience was available to guide the de. 
velopment of the test and sco 
cedure. Consequently the tryo 
vealed a number of Ways in which, it is 
felt, the test and testing procedure could 
be improved. The following suggestions 
concern improvement of the test with 
respect to the breadth of information re- 


flected in its scores, and the reliability of 
measurement. 


ring pro- 
ut has re- 


1. If one reads 


a sample of responses to the 
In-Basket Test, he 


will probably get the impres. 
sion that much more information about the ex. 


aminees is contained in the responses than js 
reflected in the scores reported. The Scoring 


РЕТ. 
procedure requires the scorer to dec 2-4 pedet 
several types of response a pase aay 3 
resembles, and this involves throwing edt 
good deal of information, The first PESE the 
tion, therefore, has to do with exte ie гау 
breadth of information obtained fron the pro 
by suitable revision and extension of 
cedure for evaluating the responses. 
One revision or extension of the = 
tem proposed here is to give iios ee 
freedom in awarding or subtracting Lae with 
exceptionally good or poor eee than the 
respect to categorics of behavior ot A үпү being 
one for which the problem is dgio yroblem 
scored, For example, in a response ta Sul ex- 
primarily intended. to measure P indicates 
aminee might write something VA of stand- 
outstanding ability to make effective ps credit 
ard procedures. Thus he would be eat in the 
for this response by tallying a ро 
“SOP” column of his score sheet. sae 
Similarly, evaluations might be mem 
acteristics that the test was not origin: 


scoring SYS" 
greater 
ts for 


vealed in the content analysis of outcome 
ments. For example, E 
tiveness seem to be revealed in 
many of the ex “ces in 
the Т of duty in offering their senan ; 
memoranda, while other students — 
in their writing a spontancous ЫШЫ i 
someone else work out a problem. ; cooper 
ments reflect a genuine attitude O rake a 
ness, it would seem desirable to! be 
procedure for recording the insano hs ү 
found in the scoring process. Anot pet b 
istic which at least oc эпаПу m sympathy 
revealed is the attitude associated dinates: 
or harshness in dealing with suborc be thot? 
In-Basket Test, in other words, nt a grea is 
of as a projective test which revea writer Deer 
more about the personality of ше! method d 
revealed by the objective scor De enti dii 
Scribed in its Appendix. pev ا‎ evaluate 
leading to modification of the test тве о 
Procedure іп ап attempt to capi is té bE 
breadth of information which o justified oe 
tained in the responses seems 10 the In- Ep 
2. The most obvious fault. of e " ЫН, 
Test in its present form is its Ed extent on, 
While low reliability might to SoM information 
Compensated for by breadth S endi е em 
Brcater reliability is certainly 3t necesst ye 
Probably could be attained withot he olor, 
decreasing breadth of information. creasing 
ing suggestions have to do with un jn 
reliability of the Iu-Basket Test SO" auge Ais 
The development of the soine BD. тезро? y 
volved making a content analysis of '' ilit 


7 
king ! 
to а particular problem and then 25 


ious degr 
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iight be e d RE reason for low reliability 
оррбвё@: tó се ап influence of military as 
ing of fac. à Ё Re judgment in the order- 
implies quodi, p sol responses. High reliability 
respet to d of the items of a test with 
ее E ii ble being measured. The pro- 
more ini S: t a research may have introduced 
A an m than was necess ry or desirable. 
ment of M i pertaining to the improve- 
the ee iy is that the homogeneity of 
procedures E increased by 5 itable analytical 
Which sins ıe major categories of behavior 
scores al to the most important subtest 
by classifvin пао etc.) were obtained 
striction des rs statements of outcomes of in- 
the aur | у the CSS faculty. Ап analysis of 
analysis for « to all the test problems by factor 
to ihe diston ane analogous method) might lead 
items into on of more valid bases for grouping 
already sonia scores. The cluster analysis 
ment that pou tends to support the argu- 
from such a ORE EREN subtests would result 
Finally asena 

у, improvement of reliability should re- 


sult fr 
Tor 2 s T 
of items | more stringent sclection and revision 


ou to the specific suggestions 
if the oe iE should be emphasized that 
evaluation кы: function as an adequate 
used ind Savine it must frequently be 
Staff. "he revised. by the instructional 
rather than involvement. of instructors, 
develop outside stall, in the continued 
be more i and revision of the test will 
Of the ions ely to ensure that the results 
Content sting will be used and that the 
of the test is appropriate. 


TI VI. SUMMARY 

ls r 3 $ | 
ОГ test report describes the preparation 
aid in os terials which were designed to 
«omm зе evaluation of instruction in the 
versity A. and Staff School of Air Unt 

sity 
i 5 ourse 


15 to A major purpose of this C 


Geny oe the administrative profi- 
„Sst mate reld grade Air Force officers- 
Топ were a for evaluation of instruc 
Support considered desirable in order to 

in curriculum 


improvement 
edures. 


anni 
ne € Д 
€ and instructional proc 


_ The test which was developed is a 
situational test which requires the ex- 
aminee to play four roles: Commanding 
Officer, Director of Materiel, Director of 
Personnel, and Director of Operations of 
a fictitious Composite Wing. The exami- 
nee is provided with background infor- 
mation about the wing and the Air Force 
base where it is located, suitable material 
from the “files,” and the contents of an 
in-basket which is appropriate to each 
role (hence the name, In-Basket Test). 
The in-basket contains letters, memo- 
randa, and other documents which em- 
body problems aimed at eliciting be- 
havior relevant to the objectives of 
instruction in the course. The examinee's 
task is to write the letters and memo- 
randa which he would write if he were 
actually on the job, to sign (or not to 
sign) letters prepared for him, or to take 
other suitable action. Scoring of these 


products is intended to provide informa- 
tion which would b 


є useful to the Air 
University in evaluating the instruction 
in the Command and Staff School. Full 
dministering and scor- 


instructions for a i 
ided in its Appen- 


ing the test are prov 
dices. 

This repo 
of a tryout oft 


rt also presents the results 
he test. The purpose of 
this tryout was to investigate some of tne 
statistical properties of the test, student 
reaction to the test and the adequacy of 
the instructions and procedures for ad- 


ministration. An entire class in the Com- 


mand and Staff School took the test in 


the third week of the course. 

The results indicate that the In-Basket 
Test can be scored with a reasonably high 
degree of reliability, but that the Vis 
form of the test is low jn content po 
pility. There is evidence. however, t e 
the content reli be improve 


by reassigning 
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"E 
ШЕ 
ЕЕ 
5l 
اب‎ 
Alo 


Int. Officer 
Stats. Officer 


ACTION COORDINATION INFO û RETURN ___ 
SIGNATURE x COMMENT YOUR FILE 1 — 
FILE: 
ms ud 2: РНЕ 
SUBJECT: 
REMARKS : 
سم‎ 
1 
FORM 71W3 Pago 2 of 7 
Fic. 1. 


sponses; by eliminating poor items and 
revising others; by increasing the homo. 
geneity of the items through use of tech- 
niques such as factor analysis. 

No criterion of validity was available 
but correlations with other measures are 
presented which seem to indicate that 
the test is not measuring the same things 
measured either by the American Council 
on Education Psychological Tes 
the CSS final course grade. There is also 
some evidence that the relationship of 
the in-basket role to the Student's area of 
greatest experience in the Air Force is a 
factor of minor importance in influencing 


t or by 


Reproduction (shortened vertical:*) of Form 71W 


3 used in the test. 


ed by 5 


sons of groups. А 
It is concluded from analysis the 
data obtained from the tryout n for 
pretest form of the test is suitable а5 
certain administrative purposes ? И 
comparisons of groups before and # 
training, but not for purpose 
vidual assessment. It is 
that steps be taken to improve 1° ig 
bility before the test is widely USC ^ pe 
gestions are presented for extent & 


al the 


s О 


| 
| 
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range А " ; ; 
E e of information obtained from the 
st. а e S > ; 
i. and for increasing its reliability 
hro ане 
ugh reassessment of scoring weights, 


improvement in the homogeneity of the 


subtests, revision of items, and changes 
© 


in the scoring procedure. 


APPENDIX 


Several sa 
ith moo problems in the form of fac- 
Солна п from the in-basket of the 
letters which А пох ате reproduced below. The 
notation M AN in memo form, with the 
cach typed on 2" V3 at the lower left, were 
this test. Tt Ie isi a. ipn specially designed for 
m the Air I to buck ip" forms used 
Fig, 1). rce, but not identical to any (see 

TI - 
Suis i POUR. is presented by à 
to correct in ic Adjutant requesting permission 
Unauthorized 1 08 of classified documents in an 
lo measure hg ing? This problem is intended 
The second с Use of Routine. 
Pilots, prepar ойе involves a directive to all 
for the ае һу the Director of Operations 
sult of a con танат Officer's signature as а Te 
the cam pam that pilots have been buzzing 
the Deer The best solutions recognize that 
Unimaginatiy of Operation’s proposed action is 
Bram for ye and that а specific positive pro- 

prevention of such violations is re- 


quire 

ed. TI 

г ; The lem: is i ds 
Oresight е problem is intended to measure 


memo- 


The thir 
Plaine a sample problem involves à com- 
have Зон a used car dealer that two airmen 
"сајле that т bad check, The candidate should 
Was inadec ше investigation of the Legal Officer 
Ser shed ds and that the Commanding Offi 
Поп before insist on a more thorough investiga- 
Pared for | signing a letter such as has been pre- 
tO measur Mm to sign. The problem is intended 
EL of Decision-Making. | 
ia letter, f problem the Wing Chaplain Pr& 
‚ for the Commanding Officer's S187 


situation is 


stud i 
y of the recreation 
measure 


Һе pr Маш" 
problem is intended to 


Uidan 
Се of Decision-Making. 


PROBLEM 1 


C 
м Ommandant 
Ом: 
E Adjutant 


Чарс, Date: g July 195% 
SS: Су " 
Ments Оттесіоп of the Log of Classified Docu- 


REMARKS: 

1. The Administrative Inspector, HQ, ggth 
AF, discovered a discrepancy between our Log 
of Classified Documents and the File of Classified 
Documents. 


2. The missing Document is Secret Letter MV 
163-002-21, dated 16 March 195% from HQ АМС. 


of the Administrative Inspector 
tion of the specific missing docu- 
ions a discrepancy between 
f Classified Documents 


3. The report 
contains no men 
ment, but merely ment 
the Log and the File o 
which must be corrected. 

4. The document described in Paragraph 2 
was burned by the undersigned after appropriate 
action on the letter had been taken. There were 
unfortunately no witnesses; the undersigned was 
new at his job and did not realize fully the im- 
portance of the action, There is no question that 


the letter Was burned. 

ion is requested to correct the Log 
s by drawing Чр a new 
Log which omits the listing of the document 
described in Paragraph 2, an! to destroy the 


present Log of Classified Documents. 


5 Permiss 
of Classified Document 


Joun т. SMITH 
Major, USAF 
Adjutant 


(signed) 


Form 71W3 


PROBLEM 2 


First Letter 
EASTERN UNIVERSITY 
CAROL, MAINE 
Office of the president June 25, 195% 
Colonel Hiram W. Goodfellow 
Commandi 
pine City Air Force Base 
pine City: 


Dear Sir: 


am writing YO 
our S 


u about 2 situation that has 
tudents, instructors, and ad- 
increasing com р Ош Uni- 


asses. They are utilizing 
jd for flying, т suppose par ly 
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and partly to earn the appropriate number of 

i urs. 
рч however, abusing this privilege tre- 
mendously to the detriment of the University. 
A great deal of flying is done over the campus, 
buzzing not only the University buildings, but 
also buzzing students when they are walking 
along relatively open stretches on the campus. 
The noise of the diving planes interferes with 
the lectures and recitations; furthermore, other 
pilots in the class seem to take it as a challenge 
and vow they will come closer than their buddy 
did just then. So far no one has been seriously 
injured or killed; however, it would scem that 
this would not be unlikely if present antics are 
continued. The University is doing everything 
in its power to cooperate with the military and 
to assist in the present emergency. We trust that 
you will be able to prevent a further occurrence 
of such unnecessary annoyances as I have referred 
to. 

Yours truly, 
(signed) Mitton Е. Jones 

President 
Eastern University 


MFJ: sbm 

Second Letter 

To: Director of Operations 
From: Commandant 

For: Action 
Five: C747 Date: 27 June 195x 
Supject: Flying Violations 


REMARKS: Please look into this an 


: а d suggest ap 
propriate action. 


(initialed) H, w, G. 
Form 71W3 

Third Letter 
To: Commandant 
From: Director of Operations 
For: Signature 
FILE: C747 DATE: 2 July 195x 
Sunjecr: Flying Violations 


Remarks: I have looked int 
in President Jones’ letter, 
gerated the situation. 


© the facts alleged 
He has not exag- 


(signed) W. T. THOMPSON 
Lt. Col., USAF 


Director of Operations 
Form 71W3 
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Fourth Letter 

To: Air Operations Inspector 
From: Commandant 

ATTN: All pilots 

For: Action 


Fite: C747 Date: 3 July 19: 


Sunjecr: Flying Violations 


REMARKS; " is 

1. It has been brought to the attention of tr 
Command that there have been тарса inum 
tions of paragraph 13, AFR 60-16, ert. 
Altitude of Flight, in the vicinity of the F3 
University Campus, Carol, Maine. 

:nformed 

2. Since this Command has not been dieere 
of these violations under the provisions PUE it 
graph 49, AFR 60-16, Authorized Deve nized 
is assumed that these incidents are E : 
by the pilots concerned as being unauth 


h 
< served by SUC 
3. No useful military purpose is served by 
conduct, 


cited reg 


AFR 36-57 
violation 


4. The attention of pilots violating 
lations is invited to paragraph 0с, 
Causes for Suspension, Serious Willful 
of Flying Regulations. 


FELLOW 
(signed) Hiram W. GOOPFF 
Colonel, USAF 


Commander 
Form 71W3 


PROBLEM 8 
First Letter 


W. Downe EAST 
THE SMILING ESKIMO! 


Pine Plaza West 


Pine City, Maine 195% 
PERSONAL Jus i 
Colonel Н. W, Goodfellow 

Commanding Officer 

Pine City Air Force Base 

Pine City, Maine 


an k 
Soldiers were Sgt. Myron О. Jones a pen 
Herbert K, Snyder. They each gave haw” е 
for $203.98, drawn оп the Nationa е we 


THE IN-BASKET TEST 


pu pay — for May. The checks have been 
asked me di cd “Depositor Unknown." They 
ЖАПА» ot to report that they were out of 
Tt m, although this was also true. 
Disger pde e been sold to Sgt. Homer н. 
imr ке кушл: m the State Registrar of Mo- 
been. sent E 115 Јопеѕ апі Snyder have 
mites Бү ныйт di according to reports given 
Î wish es 5 stil living in their quarters. 
Eier Be i would do something about this. 
[рж ep Dinger should return the car 
be mate. phe for Hs Jones and Snyder should 
passing. ack to stand trial for bad check 


Sincerely yours, 
WDE: diis (signed) W. DowNE East 


Second Letter 
To: 

9: Legal Officer 
From; Commandant 
For: Action 


File C406 
R Mb Dares w July 1955 
EMARKs: Я 
thils let, Please draft an appropriate reply to 
S letter, for my signature, 


Form TW3 (initialed) Н. w. G. 


Third Letter 


yist COMPOSITE WING 

C496 PINE CITY, MAINE 
ѕ July 195¥ 
Mr, w етем 


У . Dow 
Pine p owne East 


ine rsa West 
D Чу, Maine 
саг Mr, East: 

Your á 
ee of 28 June, 
Teferred 1 the airmen of this comm 
are sory: 5 my legal officer for consideration. we 
а ане, to learn of your experiences with them. 
Sistance = we are powerless to be of any а5- 
to the = you in this matter. Sgt. Dinger's title 
Cgistrar ee evidently clear, since the State 
of Motor Vehicles has accepted it. He 


сап 
Wie be ex ar 
Ice, xpected to pay for the same C 


criticizing the actions 
and, has been 


Altn 
a Ough we did send a Sgt. Myron о. Jones 


nd 

as ; 

here е Herbert К. Snyder overseas last week. 

aled, Ser c PronP that they were not imperson- 
ducted а 


t n 
negoti Dinger tells us that he con 
Phone fe with them by long distance tele- 
e еа day after they left here for overseas: 
them their money to Seattle, and they 
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mailed him his title. While this i i 

title. s is a highly un- 
usual way of doing business, the story aedis i 
every detail except for positive identification of 
Jones and Snyder. 


Sincerely yours, 


(for signature of) Hiram W. GOODFELLOW 
Colonel, USAF 
Commander 


PROBLEM 4 
First Letter 
To: Commandant 
From: Chaplain 
For: Signature 


Fine: C525 Date: 2 July 195X 


SuBJECT: Community Facilities for Recreation 
REMARKS: 

1. In view of the projected expansion of this 
base, I feel that recreational facilities in Pine 
City will soon become a matter of critical con- 


cern. 


2. I have prepared the attached letter for your 


signature. 1 sincerely hope that you will be able 
and willing to speak to the Ministerial Associa- 
tion on this matter. Of course, if you so desire, 
I should be glad to do so in your place, but I 
sincerely believe that your personal attention to 
this matter would insure the success of this 
project. 
S. Y. SOLE 

Major, USAF 
wing Chaplain 


(signed) 


Form 71W3 


Second Leiter 
7151 COMPOSITE WING 
PINE CITY, MAINE 
C52 و‎ July 195* 
Simon T. pureheart, Chairman 


The Rev. Mr. 2 ds 
Association 


pine City Ministerial 
Old North Church 
pine City, Maine 


Dear Reverend Pureheart: 


probably seen th 
he faciliti 


Jashington that t 
p ore с are to be markedly expanded 
W have in the past enjoyed enerally har 
v with townspeople. 


monious 


°8 NORMAN FREDERIKSEN, D. К. SAUNDERS, AND BARBARA WAND 


that have been set up in Pine City to provide 
good, wholesome recreation and diversion for 
our неп during off-duty hours. However, the 
doubling of the base complement which will 
occur over the next six months poses new prob- 
lems. - 

I am therefore writing to you as head of the 
Ministerial Association to ask that your group 


T йай йе ой 
so as to help provide increased naiue Drar 
duty activities. I shall be glad to spea e 
your group if you should desire to invite 
do so. 

Sincerely yours, 


Hiram W. GOODFELLOW 
Colonel, USAF 


(for signature of) 


consider what plans the churches might make Commander 
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Forced Choice and Other Methods for Evaluating 
В о 
Professional Health Personnel 


SIDNEY H. NEWMAN, MARGARET А. HOWELL 
United States Public Health Service 


AND FRANK J. HARRIS 
Operations Research Office, The Johns Hopkins University! 

clusion that the forced choice technique 
T was undertaken 8 com- n by hae Deparment of the 
with other ıe forced choice technique i т ns кы, Le. = for use 
performance задев for evaluating the "i cw gars d ea = i 6 s setting. 
sional е of commissioned profes- ; er ev a = Y^ he pae choice 
States РГА personnel in the United *& qs с ae ce pape 
of the off м lealth Service. It is a part озү Lus ص‎ 
Program ics Ling апа. lu sonnel oe by Sissi (19) УЫП 

The scribed by Newman (7). s , Teparii E G A Ов 
organi Public Health Service, the major Federal (13) and the йиш 9 P 
nization responsible Ws D ea of the Department of the Army (18, 14) seemed 
to indicate the usefulness of the tech- 


Natio 
i n, employs 
Ae eee d approximately 16,000 Civil Serv- 
and 3,000 commissioned officers. nique in а commissioned personnel 


INTRODUCTION 


le c 
so, COmmissi ч 
service is одос oflicer component ОЁ Ше ер 
Onnel j omposed of carefully selected pet system. , ` m" 
the heal lics and in An Experimental Efficiency Report, 10- 
Nike : 3 n А 
Лигу dentistry, corporating forced choice items and 
ar ‚ yeter- ۹ a ‚ H 
jd med асу, veter other evaluation materials not in use in 
se у 
à TC À Y and 
health, =i hold clinical, research, public the Service 1n 1919, was designed a 
Spi administrative. positions in Service distributed to the supervisors of active- 
The effectiveness of the 


h 
-OSpitals. 


Te 
опа] oll 
Orcigy 


research oo duty officers. t 
agencies а! forced choice technique 15 here com 


Outpatient clinics, 
with that of the more conventional 


ices, other 
d » other government: 
! countries governmental 


ле 

° perf 9 

Es is sane ae evaluation of commissioned pared p 

н : ; i ici i Б 

E periodic eflii- eyaluation methods included in the Ex 
ort. It is anticipated that 


Sery; Сротці 
Vice’ ng. In fof ртом е 
c an effort to improve the perimental Rep 


will contribute 


C's 
Ratio Perform; 
я n o mance-rating system, an investi- 2 
тесе Was од Гог evaluating job perform- the results of this study LE. 
is Ure in ошап in 1949. A review of the (о the literature on performance evalua 
Prego,” din | performance-rating ficld revealed n and in ра ticular, to that of forced 
ang. Ê un had been reported then, sat GORE = wn The findings may 
Yi то LI kinds of highly trained scientific choice methodo ову. тре 
leq and m personnel employed in Ше also be of special interest to = а 
РИ yar ares нле service health departments, hospitals, ! esea 
tions, ОГ other organizations em- 
i utio 
M А а sonnel engaged 


Р Consi 

a Sidera; 

Nee fen апай of various perform 
ی‎ ation methods led to the con- 


: 
i Pong, 

alth c ly wj 3 T 

1 Servie Ше United шев Public 
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health and disease. In addition, this 
study calls attention to some of the prob- 
lems implicit in the evaluation of rela- 
tively small numbers of employees en- 
gaged in a variety of specialized profes- 
sional activities. 


PROBLEMS 


An investigation of performance-evalu- 
ation methods was undertaken to answer 
the following questions: 

1. Is the forced choice technique an 
effective method for measuring the per- 
formance of professional health person- 
nel in the Public Health Service? 

2. How does the forced choice tech- 
nique compare in validity and reliability 
with more conventional methods of per- 
formance evaluation? 

3. How do factors such as the charac- 
teristic evaluated in criterion ratings, the 
administrative level of the supervisor 
completing efficiency reports, and the 
grade of the ratee affect the validity of 
efficiency reporting? 

4. What combination of efficiency- 
reporting methods optimally predicts 
the performance of personnel in the vari- 
ous professional and occupational fields 
of the Service? 

In addition to the major problems of 
the study, it was possible to compare the 
validity of the Experimental Report with 
that of the Officer’s Progress Report (the 
efficiency report in use in the Service in 
1949) which had been completed under 
operational rather than experimental 
conditions. 


MATERIALS 

Criteria 
The criteria of performance within 
the Public Health Service consisted of 
20-point graphic rating scales used for 
the evaluation of each of the following 
factors: Work Performance, Administra- 


HOWELL, AND F. J. 


HARRIS 


tive Ability, Personality (Personal Quali- 
fications), and Overall Value to the 
Service. Instructions for each scale re 
quested raters to compare the ТЕЕ with 
a typical group of personnel having simi- 
lar duties and responsibilities. A rating 
of ı was used to designate the least a 
fective ratees, and a rating of go was usec 


to designate the most effective. 


Experimental Efficiency Report 
This Report was divided into e 
following four sections, samples of whic 
may be seen in the Appendix. 
Section I—Forced Choice. 
consisted of 5o tetrads adapted 
items developed by the Departme! 
the Army (10)? Each tetrad was rip- 
posed of four words or phrases pus кт 
tive of job performance OF um 
qualifications from which a aicut 
was to select (a) the one most dese ; the 
and (b) the one least descriptive 9. загу 
individual he was rating. А pu 
investigation had shown that the publie 
tetrads adapted for use with ced а 
Health Service personnel wr 
promising number of scorable ? 
tives (9). 


This part 
from 
nt O 

com- 


a list of ten major work areas П 
Public Health Service from 
supervisor was to indicate the p 
mary job function, The super!’ 
then requested to rate, on û " 
scale, the quality of the ratec Sp 
ance in this function. "TIT 
Section  III—Personal Qualifica ing 
This section consisted of ten-point " 


[so 
3 боо]! pe 
scales for the evaluation of eight Р » 
reactio 


m 
s and er 


15. 


n 
to 


ality characteristics such as 
criticism, freedom from bia 


е 
pers 

° Appreciation is expressed tO yos ^ offic’ 
Research Branch, The Adjutant Gen these р 
Department of the Army, for makin 

terials available. 
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um ability to work with others, 
dienes hes on own responsibility, and 
necessary = persistence in performing 
ssary work. 
p 1 V—Check List. This part con- 
IS ont E statements which were to 
M Ше ees as applying or not applying 
Newer The statements concerned 
[юы Ipsi баа и 
ship, н organizing ability, leader- 
related ml and other characteristics 
ena "E performance. In the de- 
Statements м the Cheek тањ em 
from "us наб ‘been extracted verbatim 
Progress Pep sections of the Officer's 
ыо port, placed in 12 logical 
stone's a sorted according to 1 hur- 
арреагїп manon of the method of equal- 
statement оч i> 10) Ак = 
ма I comprising the final check list 
bility а baie showed the least varia- 
and wi е semi-interquartile range) 
perform: were deemed most relevant to 
ance. 


Oficer 
fficer’s Progress Report 


Sa А 
ен of parts of the Progress Re- 
sed i ae EN 
ficienc in 1949 to obtain pu 


nel y ratings on commissioned ре 
The 


Dace icem in the. Appendix. 
T n contains two types of 
(Sega 
u E Scales. Т his section consists 
"hg such jm rating scales for evaluat- 
Profession; ac as judgment, genera 
assigned de knowledge, proficiency | in 
d depen е ier ic tact, initiative, 
а papacy The scales are sp 
M | nii ae odd number values 0 
и Dh ае being assigned (0 the 
SS are aver each scales values from al 
tig, "ative м to obtain а total score 
ha i" the p omments. Several ques 
ре Tative ¢ rogress Report also elicit 
Mun a omments concerning à ratee $ 
псе, A method for scoring these 


of 


comments was developed (8). Its use 
involves assigning to a comment in a 
Progress Report the scale value of a 
matching comment in a scoring manual. 
The total score for the Narrative Com- 
ments is the average of the scale values 
of all comments in a Report. 

Total raw scores from the Narrative 
Comments and the Rating Scales are 
separately converted to standard scores 
on the basis of norms established for 


each officer grade and profession. 


COLLECTION OF DATA 


Criterion ratings were obtained dur- 
of the 54 Public Health 
in the United States, 
als, 10 regional of- 
laboratories of the 


ing 1949 from 45 
Service installations 
including 14 hospit 
fices, 8 divisions, 8 
National Institutes of Health, and 5 
other installations such as outpatient 
clinics. Nine stations Were excluded from 
the study because of practical considera- 
tions and the small numbers of possible 


ratees at most of these stations. 


Ratings were ob 
by a staff represent 
method of rating an 
Officers who worked together, regardless of pro- 
met in groups and rated cach 
f the rating factors, which 
cach officer was pro- 
rs at his station. 
officer, excluding 
did not know, by plac- 
twenty- 


that they were to be 
purposes only. The lower 
ers (the equivalents of the Navy 
full Jieutenant) were rated on 


ensign through 
i es and 

1 public Health Service ойт Ed 

iheir Navy sivalents ave given below: 
i Tay 
public Health Service e 
is Ens A 

unior assistant EID at 2 
oe Lieutenant 
ваи ваї Lieutenant commander 
ier Commander 
Led Captain 
pirector 
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one scale and the higher grades on another in an 
effort to reduce irrelevant grade-associated fac- 
tors which might affect the ratings. 

One month after the collection of critcrion 
data, copies of the Experimental Efficiency Re- 
port, directions for completing, and a schedule 
for designating supervisors to mark the Experi- 
mental Reports were mailed to the officers in 
charge of the installations which had been 
visited for purposes of collecting criterion ratings. 
Two independent Reports were requested on 
each ratee, one from his immediate officer su- 
pervisor and another from cither the officer in 
charge or his representative. 

The Officers Progress Report is requested 
annually for all officers at the Full grade (Na у 
lieutenant. commander) and above, and semi- 
annually for all officers below this gradc. For 
purposes of the present study, one Progr 
Report was selected, where possible, for h 
officer on whom an Experimental Report had 
been completed. The Progress Reports selected 
were those completed within six months before 
or after the Experimental Reports. In most 
instances, this time control resulted in a match- 
ing of the two Reports оп ratee’s Service pro- 
fession, grade, corps, and station. Progress Re- 
ports which did not match the Experimental 


Reports on these factors were eliminated from 
the study, 


DEVELOPMENT OF EXPERIMENTAL 
REPORT SCORING Krys 


Designation of Occupational Groups 


for 
Ttem Analysis 


The diversity of professions and job 
functions within the Public Health Serv- 
ice necessitated the designation of sep. 
arate occupational groups for which Ex- 
perimental Report scoring keys could be 
developed. An early analysis of the Ex. 
perimental Report showed that items 
scorable for various ratee professions did 
not appreciably overlap (9). In view of 
these considerations, the following fac- 
tors were controlled in designating 
groups for item-analysis purposes, 

Criteria. Intercorrelations among 
scores on the four criteria (not shown in 
tabular form) revealed that Work Per- 
formance and Personality produced the 
lowest intercorrelations, ranging from 
50 to .72; all other correlations were 
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higher, ranging from .74 to .92. The de- 
cision was made to use only the two 
more independent criteria, Work Per- 
formance and Personality, for purposes 
of item analyzing the Experimental Re- 
ort. 
Station. This factor was controlled by 
type of station. Stations were classified 
into three groups according to their x 
jor functions: (a) medical care, furnishec 
in installations such as hospitals and 2 
patient clinics; (b) public health eti 
carried on in regional offices and in suc i 
the Bureau О 
per 
Na- 


divisions as those of 
State. Services; and (с) rescarch, 
formed in installations such as the 
tional Institutes of Health. anl 
Profession of rater. Among mel 
care personnel (in hospitals and yr 
tient clinics), where the number of к 
terion ratings permitted, profession se 
rater was controlled by using only n 
ratings performed by members pona 
ratee’s profession. For nurses work!" 
in medical care, two groups ofr 
used: physicians and nurses. In t 
lic health and the research groups 
lished by the station control, rat i 
all professions of raters were usec ne 
cause of the small numbers of pe ic 
in any one profession. Further, ane 
health and research personnel are jsors 
ally given efficiency ratings by super”! put 
working in the same functional ar care 
not in the same profession. Medica eive 
personnel, however, not only Te the 
efficiency ratings from supervisors p 
medical care field, but frequently [ 
those in the particular profession 9 
ratee, 0 
Profession of ratee. For pupa ed 
item analysis, this variable Was oon the 
among medical care personnel 
establishment of separate groups s 16 
ing to the three major profession nurs” 
resented—medicine, dentistry, ап 


aters were 
he pub- 
, estab- 


ngs by 
be- 


d- 


accor 
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TABLE : 


MEANS AND STAN i 
AND SrANDARD DEVIATIONS OF CRITERION SCORES ІХ EACH OCCUPATION. 
Group DESIGNATED FOR ITEM ANALYSIS SENSUS 


-— _ Work Performance Criterion Personality Criterion 
cupational Group N 
No. b No. 
— Ratees M SD Ratees® M SD 
Physici AE ы Е 
Physic 3 ny 158 12.42 2.92 150 11.77 2 
Public kd d 161 12.5 2.7 I н 2. 
ire h alth personnel (1) 90 13 3: чк ы Bie i 
Ps honit personnel (2) 88 13-60 2.34 100 кү E 
Resin personnel (1) 66 13.88 2.5 | 2 
> rch personnel (2) 65 1379 Ж "d 32.56 à 
56 13.28 2.80 56 5. 00 3. 
55 13.42 2.64 55 12.84 A 
92 12.44 2.17 Q1 11.51 25 
бо 13.2 1.98 62 12.00 25. 


è Sm Жр 1; (2) =sample 2. 
‘In Food from matched sample 
ster pem tor, rs fi they Тай 
an job BEE OF IEE se raters felt th 
ї 
= ый ын of public health and 
by profess sonnel were not broken down 
Sons as th ио аке for the same rea- 
of Profession. specified in the discussion 
Grade on of rater. 
by the p TUN. Grade was controlled 
each ratec ро tionate representation of 
ow uud s in high, middle, and 
each of "x E groups identified within 
he Meer cost item-analysis groups. 
ber cent, th n groups Were the upper 27 
омер or e middle 46 per cent, and the 
by in a per cent of ratees, determined 
the appresa rating they received from 
Separate rudes group of raters on the 
апа Pers criteria of Work Performance 
Viu sonality. 
tin the occupational field of medi- 
e four groups were estab- 
bns nes uM of item analysis: (4) 
ated h ated by physicians; (b) nurses 
Physician nurses; (c) nurses rated by 
entist a and @) dentists rated by 
f lysis $ " physicians. Two other item- 
elds yy, Broups based on occupational 
h еы also established: (а) public 
?onne]. onnel rated by public health 
} and (b) research personnel 


а 


„ ә Care, 
Isheq 
Physic; 


су had gre: 


s did not differ significantly 
d on the Personality 
ater opportunity 


at the .os level or below. 
> than on the Work Performance 


to observe personal qualifications 


rated by rescarch personnel. 

A criterion score, the average of five 
atings, Was computed for each 
h criterion. А minimum of 
d to obtain as 
ble without 


or more r 

ratee on eac 

five ratings was require 

highly reliable scores as possi 

from the study a large number 
itted, 


numbers perm! 
alysis 


excluding 
of the ratees. Where 1 


the groups designated for item an 
were split into matched samples to pro- 


vide for cross validation. All groups €x- 
cept the dentists and the nurses rated 
by physicians contained enough ratees 


to furnish split samples. 
The number in each item-analysis 


as the mean and the stand- 
ard deviation of the criterion scores on 


each rating factor, is shown in Table 1. 
From Table 1, it is important to note 
that none of the mean differences 1n 
matched groups is significant (five per 


t level or below). 


group, a5 well 


cen 


Item A nalysis 
For purposes of it 
Experimental Report 


each ratee- The one ip 
termed the “primary Report, W 


em analysis, one 


was selected for 
Report selected, 
as gen- 
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erally completed by a supervisor whose 
administrative level corresponded to that 
of a branch chief in a division or a clini- 
cal director in a hospital. A “secondary” 
Report was available on most ratees; this 
Report was generally completed by a 
supervisor, such as the officer in charge, 
who was at a higher organizational level 
than the supervisor completing the pri- 
mary Report. Secondary Reports, al- 
though not used in item analysis, were 
scored to provide additional validation 
data. 
Section I—Forced Choice. In a previ- 
ous study, it was found that of four 
methods for item analyzing forced choice 
tetrads, the critical-ratio technique ap- 
peared the most useful in that it was 
relatively easy to apply, gave readily 
interpretable results, and yielded item 
weights which were in close agreement 
with weights derived from the other 
methods studied (д). In the present work, 
critical ratios were used to item analyze 
the Forced Choice section against the 
separate criteria of Work Performance 
and Personality. Within each item-analy- 
sis group, the significance of the differ- 
ence was tested between the percentage of 
the high and the percentage of the low 
criterion groups rated on each alterna- 
tive. In the medical groups, the samples 
were sufficiently large that it was possible 
to use a critical ratio of 1.96 as the 
standard for scoring. In the remaining 
smaller occupational Broups, an alterna- 
tive was deemed scorable if the critical 
ratio were 1.50 or greater. Unitary posi- 
tive weights, indicating that a signifi- 
cantly higher percentage of a high cri- 
terion group than of a low criterion 
group had been rated on a given alterna- 
tive, and unitary negative weights, indi- 
cating the reverse, were assigned the 
scorable alternatives in the tetrads. A]. 
ternatives which were nondiscriminating 
received zero weights. 
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scorable tetrads for each 
group. approximately the 


From the 
item-analysis 
best 20 were selected to constitute scor- 
ing keys. In addition, for each of the 
item-analysis groups which had p 
split, a combined sample scoring S 
was developed. This key was composed о 
the best go tetrads selected from those 
in which only alternatives having identi- 
cal scoring weights in the matched sam- 
ples had been retained. For the eur 
of nurses rated by nurses, only à кы 
bined sample key was developed EE 
the matched samples did pot dee 
ally yield enough scorable items 9 
separate keys. ; 

A total pm on the Forced hue" 
section was obtained by summing 2 
positively weighted alternatives. A T 
vious study indicated that pos P 
weights scoring yielded as valid pem 
as positive plus negative weights audi 
ing on three lengths of keys, one of w^ б 
was a twenty-tetrad key. The crm € 
this length of key also compared е7 
bly with that of the other two key leng 
studied (s). - sonc} 

=з II and III—Job dique 
апа Personal Qualifications. Since dur 
two sections consisted of ten-point не. 
scales, the same methods of item апа im 
were used for both. First, the pt 
tory capacity of the scales was abet 
by testing the significance of the anc 
ence in the mean scale values of bio 
of low criterion groups. Of je 1 
number of critical ratios computen a 
the Personal Qualifications scales, об 
sidering all item-analysis groups gt 
per cent (113 out of 160) were pap 
at the .о5 level or below. On the ite 
Job Proficiency scale, seven of ет d 
analysis groups yielded significa” yee 
ferences (о; level or below) Бе s 
upper and lower 27 per cent ӨШ 
one or both criteria. ., each 

The raw scores of all ratees 1" 
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M шарор were used to develop 
With ho و‎ ш one-half sigma units, 
Stanine of ең E каше equalling а 
sulting Es ain The stanine scoring re- 
bu. pert samples was so simi- 
bined to ^ г раена дна ie LEO 
stanite вс; € | greater stability to the 
Proficiency м Cros validation of the Job 
sections : аш Personal Qualifications 
of the бад. та ed unnecessary in view 
tehêl ss stency in scoring from one 
Thet а mple to another. 
Dune on score on the Personal Quali- 
stanines m was the average of the 
he “ано om the eight rating scales. 
Stanine settee о Proficiency was the 
in his prim: = ] the rating given a ratee 
While Pg job function. . 

to treat se dius, have been desirable 
lions T arately each of the 10 func- 
tion, this ben the Job Proficiency sec- 
the small bi: not possible because of 
each o i of ratees performing 
Поп were on. All analyses on this sec 
Суре of we Lacie without regard to the 
Чоп foe involved. Specific job func- 
ENSE, in “т. completely masked, how- 
Sstablishin lew of the method used in 
SXample : item-analysis groups. For 
lealth le the 179 ratees in the public 
Work hoa, who were rated on the 
Ven ratiy rmance criterion, 138 Were 
“lency ngs on the Experimental Effi- 
descrip €port in the primary function 

ed as « a 
t Specialized GPération in a technical 
Se ection ro ublic Health. program. 
i те comput + List. Critical ratios 
hi he differen to test the significance 
ar I and of | nce in the percentages of 
1 each alte E criterion groups marked 

rnative, Items which discrim- 


ot 


thi 


Sroups, 42.3 per cent (186 


440 

of 

* leve] the ratios computed rea 
Of significance; the scorable 


ched 


EL 7 


лена, понеке тсе aut edy dà 

à g arious item-analysis 
groups. 

Considering the separate item-analysis 
groups, the number of scorable items 
was such that it was feasible to develop 
scoring keys against the Work Perform- 
ance criterion in only the medical, pub- 
lic health, and research groups, and 
against the Personality criterion in only 
the medical groups. Since these were 
split groups, the decision was made to 
include in the scoring keys only those 
items that reached the required level of 
significance in both of the matched sam- 
ples rather than to develop scoring keys 
for the separate samples. 

The total score on this section was the 
sum of all positively weighted items 
(those characteristic of a high criterion 
group). 

RESULTS AND INTERPRETATION 


Comparison of Forced Choice 


Scoring Keys 

The Forced Choic 
Experimental Reports 
split sample item-analysis group. 
scored by three keys: (а) self scoring, de- 
veloped from item analysis of the sample 
being scored; (b) cross scoring, developed 
from item analysis of the matched sam- 
and used for cross validation; and 
combined sample scoring, based on 
ves that gave the same scoring 


oth samples. 
coefficients ba 


e section of the 


from each of the 
s was 


ple 
(c) солі 
alternativ 
weights in b 
validity 
type of Forcec 
in Table ? 


ata OI 
econdary Repo 


Reports use 1 
the validity coefficients 
d that the 


sed on each 


1 Choice key are presented 
for the matched sample 

1 the three scoring keys 
rts as well as 


d in item 


8 s. 
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TABLE 2 
FORCED CHOICE VALIDITY COEFFICIENTS FOR MATCHED SAMPLE GROUPS" 


Work Performance Criterion 


Personality Criterion 


m x T - MT * 1- 
Occupational Group No. eat pow соп. | Жү, E ges coni 
р cor- cor- bine quae GOD дне STR 
Ratees ing ing Scoring Ratees ing ing Se ows 
Primary Reports RE: 
- puc и | ‚69° 
Physicians (1)* 158 -67 -65 -66 159 б ED 
Physicians (2) 161 .58 «55 -56 | 161 56 58 
Public health personnel (1) go „62 50 «S79 90 50 E 
Public health personnel (2) 88 ‚бї E -59 100 59 "ba 
Research personnel (1) 66 «67 .63 -09° 60 55 Gh 
Research personnel (2) 65 67% .57 ‚02 67 56 
Median r 65 .56 ‚бї “58 +56 ee 
Secondary Reports = 
Physicians (т) 120 .67 .66 .67 120 .70 -60 ae 
Physicians (2) 123 ‚70 .67 .68 123 163 66 D 
Public health personnel (1) 64 .38! .34 ‚зо 71 247 m ur 
Public health personnel (2) 63 +47 44 „5 08 „42 .50 “30 
Research personnel (1) 51 до Кш 37 53 .04 .55 8 
Research personnel (2) 56 .53 .52 .57 58 ‚50 ‚бо sd 
Median r $0 48 ‚БҮ وچ‎ 38 59 
x АП rs not marked are significantly different from zero at the „от level or below. 
7 15 significantly different from zero at the .os level. 
s (1) =sample т; (2) «sample 2. 
i r on self scoring is si nificantly higher at the .or level than ғ on cross scoring. 
‘ron combined scoring is significantly higher at the .os level than r on cross scoring. 
r on self scoring is significantly higher at the :05 level than r on combined scoring. 

Ж. а за : ferences 
two differences significant at the 05 tions and from the tests of Louise? 
level or below occurred. These were on in validity from one type of dim ia 
primary Reports, Personality criterion, another. Tt should be noted that ind 
in the comparison of physicians (1) апа differences in validity by type of sape 

PLI еи А 5 А 
(2) on the self (rs = „71 vs. 55) and the appeared to decrease when scoring ine 
combined scoring keys (rs = .6g уз. 54)^ were applied to Reports (seconda) is 

а so а ana 

Table 2 also shows that the self and dependent of those used in item 2? m 
the combined keys, both of which тер- Only one significant difference 1? di 
resent the use of scoring keys with item- lidity by type of scoring occurred on 
analysis groups, tended to produce higher secondary Reports. ple) 
validity coefficients than did scoring keys Correlations (not shown in the ta vj 
used with independent samples estab- 


lished for cross validation. This trend is 
apparent both from the median correla- 


* Here, (1) = sample 1; (2) = sample 2, 

“For purposes of testing the significance of 
the difference in rs, rs were transformed to zs, 
Tests of differences reported in this paper in- 
volved independent samples and the same sar 
with one array in common (4. p. 
45. 47, and 49). 


nple 
121, formulas 


ae K 
Ar 
among the three types of scoring 5 
were high. Considering coeffici reports 
on both primary and secondary ке] 


Е m 
° Я кы һауе jn 
“Median correlation coefficients reade" » 
presented. in tables merely to aid the х і 


"i € 
hey are P n 


"E 
tics SINC’ ve 

qe 
groups 1 і 


pu atio! 


observing trends in the data. The 
tended to be precise summary statis 
assumption that the various officer 
samples drawn from a common pOl 
not warranted. 
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ТАВ 


VALIDITY COEFFICIENTS For ALL SECTIONS 


VALUATING PROFESSIONAL HEALTH PERSONNEL 


LE 3 
or THE EXPERIMENTAL EFFICIENCY REPORT* 


m r Work Performa " iteri 
Ges upational а mance Criterion Personality Criterion 
Group No. FC JP Q No 
Ratees 2 7 P CL bre 
= Ratees adi: Ratees FC JP РО CL 
Primary Reports 
ysicians |: " | 
› 3 5 5 E "n 
Public health 319 „OI .62 .58 «54 320 .62 до 5o 6 
R personnel 178 <8 ` " 
шн personnel 131 63 ч s 35 199 ji 18 p 
Nurses rated by ` y E ч 3 чә 85 p» dun 
nurses S | 
т IIT .42 3 3 
i: P iod 42 EL 39 111 43 42^ — .35 
в у 
Dentis 92 44 28 43 
ae .28 -43 от .37 Я 3 
sts бо E -49 «55 50 62 2 n 38 
so 5: 55 S 2 .35 1 .40 .33 
Median r 56 47 46 54 50 26 36 
BL ad _ +3 2 30 4° 
Secondary Reports 
ans р - a e p d pt E 
iiir 243 .07 .58 3 5 
so health per- » М 3 - “ч bi " чё = 
sonne 
ма F 1 27 «an î 37 240 130 50 +30 +33 
personnel | 107 iS a aE c4 1i1 5S .32 39 
B es rated by Өз 4o nt 35 Б +53 sa d 
Doa "T M. 72 54$ T 27 
42 „бб 255 ‚48 64 43 .50 so 35? 40 
Media 
280 п i so 42 .39 .55 „5а 31 34 +53 
* All rs иа ге еа 
Il rs not marked are significantly different from zero at the .o1 level or below. 


at the .05 


"rds signifie А 
ignificantly different from zero 
nificance. 


° р 
r does n 
wes not reach the .os level of sig 


the . 
is NE салтат between the self 
Self and кн keys was .92, between the 
the cross z combined, 96, and between 
the self | and the combined, 96. Both 
related the cross scoring keys cor- 
Since med with the combined key 
tetrad e latter was composed of the 
samples, scored 1n both 
the = be expected that the self and 
higher keys would yield the 
Were Ж idity coefficients since they 
9wever to score item-analysis Reports 
Plieq - i when the self keys were ap- 
Ports de keys to Experimental RC 
Analysis E of those used in item 
gly lite ле validities exhibited surpris 
Possible "i decrease in size. Out of ?4 
comparisons of the self and the 


alternatives 


level. 


only three (12.5 per cent) 
ant at the .05 level or be- 
low. Although cross-validation data were 
not available for the combined keys, it 
seems likely that in successive samples 
they would have greater stability than 
keys derived from item analysis of Re- 
5 completed on a single sample. For 
this reason, subsequent discussion of the 
Forced Choice section of the Experi- 
mental Report will be based only on 
data from the combined sample scoring 

ease the reliability 


keys. In order to incr 
based on these keys, 
ombined. 


of the statistics 
matched samples have been тес 
erimental Report 


y of the Exp 
validity coef 


Validit 
Table 3 presents the 
cients based on the Forced Choice (FC), 


cross keys, 
were signific 


port 
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Job Proficiency (JP), Personal Qualifi- 
cations (PQ), and Check List (CL) sec- 
tions of the Experimental Report? To 
facilitate interpretation of the correla- 
tions, the factors which may be influenc- 
ing variations in the data will be indi- 
vidually considered. 

Report sections. From the median cor- 
relations in Table 3, it appears that the 
validity coefficients for the Forced Choice 
and Check List sections were higher than 
those obtained for the Job Proficiency 
and the Personal Qualifications sections. 
More specific comparisons of those 
Experimental Report sections which 
showed significant differences in valid- 
ity may be seen in Table 4. 


Out of the 108 possible comparisons of Ex- 
perimental Report sections, 36 (33.3 per cent) 
were significant at the .05 level or below. 


In 
27 of the 36 si 


gnificantly different pairs of co- 
efficients, the Forced Choice section exhibited 
higher validities, In only one instance was the 
validity of the Forced Choice section significantly 
lower than that of anoth 


: er section, The Job 
Proficiency and Personal Qualifications. scales 
produced the lowest coefficients; in 15 inst 


ances, 
each of these sections yielded a validity co- 
efficient which was significantly lower than that 
of another section. The Job 


c c Proficiency. scale 
in only three instances and the Personal Quali- 


fications section in only two instances produced 
validities which were significantly higher than 
those of other Report sections. The Check List 
in four comparisons yielded a significantly 
higher coefficient than another section, and in 
five comparisons, a significantly lower coefficient, 
In four of the five instances in which the Check 
List produced a lower Coefficient, it was com- 


pared with the Forced Choice section of the 
Report. 


In general, the Forced Choice section 
gave the highest validity coefficients. 
Comparisons of validities on the remain- 
ing three sections of the Experimental 
Report yielded relatively few significant 
differences although the Check List, 


* Reports in the dental group were scored by 
the key developed on physicians, An exploratory 
validation study showed 


[ ом as high validities for 
dentists as for physicians when the medical key 
was used to score reports in both groups. 
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where available, tended to produce some- 
what higher validities than did the Job 
Proficiency and Personal Qualifications 
sections. 


TABLE 4 


: THE EXPERI- 

COMPARISONS OF SECTIONS OF THE [e ү} 
MENTAL REPORT IN WHICH VALIDITY E 

CIENTS DIFFERED SIGNIFICANTL 


Primary Secondary 
| Reports Reports 
i Work Performance 
n. Кой Criterion 
rou 
i Report Report 
ih Sections 
Sections pesi 
Compared* Compare 
pi н 5 T€ р” 
Physicians FC s. CLe | FC vs. J 


JP уз. CL 


Public health per- 
sonnel 


Research personnel 


Nurses rated by 


ans 


Dentists 


аралынан ана 


Physicians 


Public health per- 
sonnel 


Research personnel 


vs: J b 
Nurses rated by FC vs. ЈР° ЕЕ уз. PQ 
nurses Pe 
ЕС vs J 

Nurses rated by at 
physicians EET 
= JP vs- РО 
Dentists 1i 


„ ® The rating section on which m 
ity was obtained is listed first. ctions Coe 
^ Validity coefficients for the a level OF 
pared differ significantly at the -° 
low. ions 
° Validity coefficients for the к 
differ significantly at the .o5 level. 


е 
comp?” 
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“SSIONAL HEALTH PERSONNEL 1 
" TABLE s 
OMPARISONS OF OCCUPATIONAL GROUPS IN WHICH Vauipity С 
E DIFFERED SIGNIFICANTLY ; EL C LE 
Primary Reports 
Е Work Performance Criterion 
aeons Personality Criterion 
Groups Compared Report 
are per Section Groups Compared? Report 
тоты 3 ection 
8 rated by nurses FC | ysici 5 
med s maes ЕС Physicians vs. Nurses rated by nurses ЕС 
P.h 4 Nurses rated b; s 
‚ h. personnel pe Dentists. y phys. FC? 
pe 
1 Es Res. personnel i 
m Res. personnel ро 
. personnel Por P.h. personnel vs s 
Res. personnel vs rated by nurses POS pemensél УЕ Nurses Аййй burps Ree 
5. rated by nurses FE Res. persi el vs Si 
s rated by phys. | ЁС“ es, personnel ve Nurses rated ру рде | FG 
Dentists А FC 
: Secondary Reports 
"hysicians 
ans SS "meas T "= 
pum I Physicians уз. P. h. personnel FC 
ded Nurses rated by phys. ЕС 
ted by nurses FC P. h. personnel Р 
ре! um ses +, Res. personnel ТР? 
ted by nurses nbd Nurses rated by nurses jp» 
ed by phys. Im Nurses rated by phys. ype 
personne re р. . h. personnel PO 
sonnel р ез. personnel e 
Dea rated by nurses PO? Nurses rated by nurses poe 
ntists a DEG rated by phys. PO . Nurses rated by phys. РО» 
"oR h. rsonnel FC Dentists ys. Nurses rated by nurses Wee 
Nurses rated by phys. ye Nurses rated by phys. Јр 


isted first. 
igni antly at the 
significantly at the 


idity was obtained 
ompared differ 
ompared differ 


„от level or below. 


a 
The i 
re үү онр in which the higher v 
.о$ level. 


ity соейїсїө! 
alidity c ients for the grou 
lity coefficients for the grou 


of the significant comparisons, the lower coef- 
ficient occurred in опе of the two nursing 
groups. Coefficients in the public health, re- 
search, and dental groups tended to be of about 
the same magnitude, differing in some instances 
from the two extreme groups of physicians and 
nurses. No significant differences were found 
between the public health and the research 
groups; only one significant difference occurred 
in the comparisons of dentists and research per- 
sonnel, and two in th f dentists 
Ith personnel. 


ee group. From Table 3, 
also ee en of the difference was 
diente aktai etween the validity coeffi- 
group ies from опа occupational 
Yielding si another. л he comparisons 
marized j gnihicant differences are sum- 

n Table 5. It should be men- 


tione 
d th: 8 
that tests of differences were only © comparisons 0 


Made 
tional between independent occupa- апа public hea 
ig groups. Th А ursin roups кы. б 
Which оу ка jd буй ore pa The higher validities in the physicians 
n erlapped in membership, were Ы i ET 
9t compar ё group are perhaps due to factors intrinsic 
Tita pared since the amount of com- 5 Eis f 
Utational ; А in the work situation. Such factors may 
al work involved did not seem rk situa ely greater оррої- 
be supervisors relatively greater ОР} 


liffer- 


Warr 

а é 

nted by the relatively small ¢ sanity B © 
1 


ences i P s 1 
instances Prata p e gm medical personnel, particu 
Out of еп the two gronp™ or lower grade officers under close super- 
ane 5 Ton m group comparisons Low vision, and to develop du x T 
xt E 5 SERA А e pet ) below. ards and rating РЕР Кс. Sion 
Comparison striking that in 33 of the significant cians constitute the э T к 
gher ds the medical group produced the group in the Public Hea ervice. 
did idity coefficient, while in no а Another possible explanation of the 
n 


it 
Produce a significantly lower one Ir 
8l у 
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validity coefficients obtained for the 
physicians group is that, in item analysis, 
a higher critical ratio was used as the 
standard for scoring in this group than 
in the other occupational groups. How- 
ever, a recent study would seem to indi- 
cate that a stringent requirement for the 
level of discrimination of individual 
items does not necessarily increase total 
validity (2). 

Criteria. Inspection of the validity 
coefficients (Table 9) from one criterion 
to another within the same level of 
supervisor shows that on all but the 
Forced Choice section higher validities 
occurred on Work Performance than on 
Personality. On the Forced Choice sec- 
tion, as high or higher validities were 
obtained on the Personality as on the 
Work Performance criterion tor both 
levels of reporting supervisor, and for 
all occupational groups except the den- 
Lists, and the nurses rated by physicians. 

A possible explanation of the observed 
differences in validity by criteria may be 
that a supervisor, if able to ascertain 
what constitutes a "good" and a" poor" 
rating (as is the case for rating scales 
and check lists), can more ob 


jectively 
evaluate the ratee on observ 


able work- 
performance characteristics than he can 
on personal characteristics, When good 
and poor ratings are not so readily dis- 
cernible, as presumably is the case with 
the forced choice type of evaluation, the 
objectivity of evaluations is perhaps in- 
creased so that they are as valid measures 
of the factors involved in a Personality 
аз in a Work Performance criterion. 

Level of supervisor. The validity coeffi- 
cients in Table 3 may also be compared 
by level of supervisor. Since the primary 
Reports were used in item analysis, it 
is to be expected that they would yield 
higher validities than the secondary Re. 


ports. The median correlations in the 


table, however, indicate that validity 
held up surprisingly well on secondary 
Reports; the median correlations on the 
Personality criterion were even some- 
what higher on the secondary than on 
the primary Reports. Considering the 
42 pairs of coefficients which can be com- 
pared from one level of supervisor i 
another, primary Reports produced ar 
higher validity in 21 comparisons, ane 
secondary Reports produced the higher 
validity in the same number es 
parisons. The median difference 1n a 
efficients was .o7 for those йрй 
in which primary Reports pue 
higher correlations, and .08 for thos 


А ; ; е агу R€ 

comparisons in which secondary 

ports gave higher validities. 1 
Differences in validity Ьу leve 


supervisor, however, were observable 
within specific occupational нир ns 
75 per cent or more of the compe ai 
of coefficients within the public - ; 
and the nurses-rated-by-nurses E 
the higher validity occurred on —— 
Reports. Since the primary Reports “ in 
used in item analysis, this бшш i 
the expected direction but in n ар 
possibly reflects the fact that D 3 i 
mary, more immediate, "apap in 
these groups are more likely tO : sec 
the ratees’ profession than are Ве or 
ondary supervisors. In 75 per © the 
more of the comparisons УЙШ, а? 
medical and dental groups, the E 
coefficients occurred on secondary hos- 
ports. It should be mentioned eee 
pitals and outpatient clinics are pent 
istered by a medical officer, ind office?” 
services are headed by a dental anc 
For this reason, both the secondar У ian 
the primary supervisors of РИ" per 
and dentists are likely to be " or © 

the same profession as the ratees ri- 


-formet 
same as the raters who perform supe" 
y 


cent 


A » 2 А аг 
terion ratings. Further, seconda 


METHODS FOR EVALUATING PROFESSIONAL HEALTH PERSONNEL 13 


visors of physicians and dentists are 
likely to be officers who routinely review 
efficiency reports and, therefore, have 
more information available as a basis 
for evaluating a ratee than do the pri- 
mary supervisors. 
_ In general, then, Reports completed 
independently by a second group of 
supervisors produced validities that com- 
pared favorably with those based on the 
Reports used in item analysis. It is likely 
that, had the secondary supervisors been 
at the same administrative level as the 
primary, differences in validity by level 
of supervisor which were apparent in 
specific occupational groups would have 
tended to occur less often; that is, the 
validities would be more nearly the same 
in all groups than occurred in the pres- 
ent data, 
Ratee grade. А control on ratee grade 
was used both in the administration of 
— rating forms and in the estab- 
ishment of occupational groups for pur- 
Poses of item analysis. Nonetheless, since 
pig not feasible to control grade more 
b ER this factor may be operating 
) ncrease spuriously the correlations 
shown in Table 3. In order to check this 
жон; validity coefficients by grade 
computed; small numbers of ratees 
instances, 


ma * . 
ade it necessary їп some 


ho 7 : : 
RS Wever, to combine adjacent grades 
ше S 1 1 ч -. 
ch as the senior and the director. Cor- 
Table 


relations 
6 ations by grade are presented in 


T ry were a systematic factor iem 
Report 1 the criterion and Experiment 
that й май езу. ЧА would be pun 
Wo те validities based on all grad es 

ў PAK в higher Шап the individual 
hu ына, That this is not the of 
Not е seen from Table б. 1 Һеге ааг. 
їп TR to be any consistent in 
ev ài correlations as a function of ge 
el. The effect of combining grades was 


the masking of the higher validity ob- 
tained for some specific grades. In only 
one instance, їп the nurses-rated-by- 
physicians group, was the correlation for 
all grades higher than any of the validity 
coefficients for the individual grades. 


The significance of the difference in validity 
coefficients from one grade to another was tested. 
The relatively few comparisons, 16 out of a 
possible 122 (13.1 per cent), which yielded dif- 
ferences significant at the .05 level or below are 
shown in Table 6. No differences occurred in 
the public health group, and only two were 
found in the group of nurses rated by physicians. 

In the medical group, the significant. differ- 
ences tended to involve higher validities in the 
Assistant grade as compared with other grades. 
Out of the 14 significant differences found in 
this professional group, 10 occurred in com- 
parisons in which the higher coefficient. was in 
the Assistant grade, This finding may be due 
to the fact that the majority of Assistant grade 
re interns under close supervision; 
erns are experienced raters 
to observe the 


physicians 
the supervisors of int 
who have ample opportunity 
interns’ performance. 
The remaining four significant differences in 
the medical group occurred on the Reports 
completed by the secondary supervisors, Work 
Performance criterion, in the comparison of the 
combined Senior and Director grade with the 
Senior Assistant grade. The unusually high va- 
lidities in the combined Senior and Director 
have been due to the small number 
of ratees (about half the number available on 
y Reports) and perhaps to a selective 

1 in the designation of highly 
e secondary supervisors 


grade may 


primary 
factor that resultec 
experienced raters as th 


for this grade. 


Although differences in the validity 


coefficients found for the various ratee 
grades did occur, they were presumably 
1 It of certain identifiable influ- 
ences. The grade factor, as such, does 

ve been operating in 


not appear to ha | r 
systematic manner which spuriously 
yalidities based on all 


the resu 


any 
increased the 
grades. 
Reliability of the Experimental Report 
Rater Correlations be- 
tween scores and secondary 
Reports, shov , provide опе 


agreement. 
from primary 
yn in Table 7 


TABLE 6 


VALIDITY COEFFICIENTS FOR THE EXPERIMENTAL EFFICIENCY REPORT 
BASED ON SEPARATE GRADES 


Primary Reports m 
Occupational Grade Work Performance Criterion Personality Criterion 
Group | 
N FC JP РО сї, N re JP PQ cL 
Nurses rated by | Assistant 48 0640 .45 04 CES EE NA 
physicians Senior assistant 28 @ +02 -19 28 -30 329; ig 
All grades* y . 28 аз or 37 29 36 
b 
Physicians Assistant 83  .64 «71° 6 —.60 85 +62 ae 980 et 
Senior assistant 108 -54 +53 +50 “45 108 205 +35 +43 48 
Full S8 0 5 A a8 ж D A “т 
Senior & Directcr то NT .65 .57 48 71 +50 -24 D 28 
All grades sig; dn б шй sy | oe абы ae чт of 
Public health per- | Full SS .68  .63  .ss .66 so .59 -39 +39 
sonnel Senior 63 ET: „43 „4 +52 73 .бї 35 E 
All grades 178 .58 +49 «49 .54 109 +57 +290 +36 
Secondary Reports 
Nurses rated by Assistant 35 63 ET .39 35 51 —.10 +18 
Physicians Senior assistant 24  .53 -49 .22 24 -48 1589 +50 
All grades 73 +52 +22 +34 72 ELI .10 zu 
Physicians Assistant 62 69 той 688.708 62 66 [ +6 i 
5 " о 
Senior assistant бе 8 әда .42 <44 тоо .67 E 5+ P 
— Ll = = к= E 
Full so 73 52 .62 58 5o E 33 ae 
Senior & Director 31 .8o* 73° 75° 8o зї 68 51 9 
А | "е 
ll grades 243 .67 — .58 „б óo 243 .68 gx £5 
Public health per- | Full 
sonnel Senior p 16 pis p po ЧӘ з RU з S 
44 34 044 655 т «65 NE 
All grad Е 
жай 37" 47 зз И AD 139 д 39 53 
a ЧТ " я 4 1 gi 
Р а on au available cases including those in grades too small for the separate grade аЛ ал 
ог e erli 1 vi 1 
officer group. evel than the underlined r in same column within the same report ыша е 
€ r is significantly higher at the .os level i i дап the sam 
officer group. 5 level than the underlined r in same column within the same report within 


TABLE 7 Е 
ConRELATIONS BETWEEN Scores р 


ROM PRIMARY AND SECONDARY Lom 
Work P. "Wes E x E 
Occupational rk Performance Criterion Personality Criterion 
Group No. a " 
Rates ЕС JP PQ CL 4 s FC JP PQ M 
Physicians 243 .57 , 59 
Public health per- 5 51 -58 „61 243 .бо .51 .88 
sonnel 127 8 k 4 
Research personnel | тоў kd 31 s 47 139 .63 .46 s 
Nurses rated by -54 III .64 .54 $ 
nurses 9r -59 M 
Nurses rated by 39 55 91 .57 39 55 
physicians 73 +50 үз” ‚48 
Dentists 42.65  .ss “| i n EZ a ү ‚61 
" * К бо 
Шил LAN NC NM. 663 49 54 


a All rs not marked are significantly differen 


Du e 3 t from j г 
h^ r is significantly different from zero at the .os evel ee ee ااا‎ 


` the f 
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TABLE 8 


M 
RELIABILITY COEFFICIENTS FOR THREE SECTIONS OF THE EXPERIMENT. 
EFFICIENCY REPORT зс 


Forced Choice Personal i 
" Qualifications" Check List 
ccupational Group N No. 
1 - ч 
Be, some SR LUE scored 
з  Tetrads ^ MN" * ит п | Alterna- "HH Fı 
ives’ tives 
319 20 28 .83 or 80 -94 12 
178 18 30 1з lor | -83 9t 7 eU NET 
131 Б 22 180088 190 .05 9 wb 18 
"e 21 20 78 .88 192 -96 3 
2 9 23 64 .78 -94 -97 
бо 20 28 .83 «91 +90 «95 12 .86 .02 
Median ri 
Se .99 -95 -83 
Fhe ا‎ 
b The тшер Өг onu. 5 ht, with the possible stanine score ranging from 8 to 72. 
scored a ather than tetrads as the basis for computing reliability coefficients. 


na ol measure of the reliability of the 
Xperimental Report. 
Morren between Reports com- 
ranged b the two groups of supervisors 
55 or bres ж] to .65; over half were 
Nu ne pee Only the lowest correla- 
significan hup [m reach the .01 level of 
leve]. " "s it was significant at the .05 
uta be noted that the Job 
BE кы ШП ) section which consisted 
the nea тапар scale tended to produce 
indicate, dae neg As the median TS 
tended E ıe Forced Choice (FC) section 
although Pages the highest correlations, 
terion he, the Work Performance cr 
those ph n did not differ markedly [rom 
cations р, uced by the Personal Qualifi- 
tions. VE and Check List (CL) sec- 
Forced pi the Personality criterion, the 
Correlati loice section gave the highest 
two E in all instances, although the 
Ast ere. ations available on the Check 
Since po comparable size. 
Uperviso BE primary and the secondary 
tive mac were at different administra: 
Омер зы the correlation coefficients are 
eports an might be obtained between 
Visors at completed by different super- 
Ports co the same level or between Re 
9n two mpleted by the same supervisor 
different occasions. Considering 


a = А 
ctors operating to lower the coeffi- 
8 


cients, the correlations. between scores 
on primary and secondary Reports, as 
measures of rater agreement, are fairly 


high. 

Spearman-Brown estimates. From pri- 
mary Reports, scored by the key de- 
veloped against the Work Performance 
criterion, correlations between the odd 
and the even alternatives in each of three 
rating sections Were corrected for length 
by the Spearman-Brown formula. A pre- 
vious paper has reported that for Forced 


Choice tetrads, Spearman-Brown esti- 
ere fairly close ap- 


mates of reliability W 
proximations of empirical reliabilities 
(5). Spearman-Brown estimates based on 
the present data are shown 1n Table 8. 
Since the Job Proficiency section involved 
only à single rating scale, it was not pos- 
sible to compute а split-half coefficient on 
this part of the Report. | 
Considering the length of the various 
sections of the Report the reliability co- 
efficients are in the high range As can be 
seen from Table 8, the number of scored 
alternatives ОП the Forced Choice and 
Check List sections varied somewhat for 
the different i nal groups. The 
i the Personal 
the same 
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i : 5 1 = Basrs OF 
COMPARISON OF EMPIRICAL VALIDITY COEFFICIENTS WITH VALIDITIES PREDICTED ON THE ВА 
| AN INCREASE IN LENGTH OF SCORING KEY 


No. imit of 
А А Scored Empirical Estimated Limit s 
Occupational Group Section Alierna- т Validity Validity Validity 
tives — 
Physicians FC 28 91 -61è o4 
PQ 8 94 58 59 = 
EE 12 85 54 50 Sr 
Public health personnel FC 30 9t 58 4 
PQ 8 9t 49 51 A 
CL 7 80 -54 59 renes 
Research personnel FC 22 88 -65° 99 
РО 95 +39 яо E. 
CL 81 50° -63 > 
Nurses rated by nurses FC 29 .88 42 48 
» eb 
РО 8 96 -39 +40 MT 
Nurses rated by physicians FC 23 .78 E s t 
РО 8 97 43 43 = 
Dentists FC 28 .91 50 
РО 8 a .56 «50 
[eria 12 .92 551 M^ 


^ ris significantly higher at the гот lev 
officer group. 


Ъ ris significantly higher at the -05 level th 
officer group. 


* Estimated validity based on the same 
4 Limit of validity if report section wer 


mates of reliability occurred on the Per- 
sonal Qualifications (PQ) scales: al] Ty's 
were .91 or higher. Coefficients on the 
Check List (CL) ranged from .80 to -92, 
with a median of .83. The Forced Choice 
(FC) section yielded satisfactory reliabil- 
ities in all officer groups except the 
nurses-rated-by-physicians (у = 278); all 
other coefficients on this section were .gg 
or .91, with a median of .9o. 


Validity as Related to Length of Scoring 
Key 


Since the sections of the Experimental 


" s (thin the 
el than the underlined r in the same column within t 


" К “thin the 5 
an the underlined r in the same column within t 


same 


1те 


"hoice section 
number of scored alternatives as the Forced Choice 
е made infinitely long. 


ye 
Report were not equated in n pi 
fact that the Forced Choice was the i 
est section may account for its on 
validity. The effect of length of oneri 
key on validity was tested on the s dei 
mental Report sections for which vail- 
validity and reliability data were а by 
able from the primary Reports pep 
keys developed on the basis of the ' 
Performance criterion. The results 4 pre 
tests are shown in Table 9 m К! 
sents: (а) empirical validities a^ 1. 
abilities and the number of score 
ternatives in each Report sectio: 
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peated from previous tables for ease of 
comparison; (b) estimated validities for 
die Personal Qualifications (PQ) and the 
Check List (CL) sections, based on an 
Increase in length of these sections to 
that of the Forced Choice (FC); and (c) 
the maximum validity that theoretically 
could be obtained on each section if it 
were made infinitely long (1, p. 166). 
И From Table 9, it may be seen that the 
es Choice section produced the 
st empirical validity in all but one 
occupational group, the dental. Theo- 
Tétically increasing the length of the 
Other Report sections to that of the 
Forced Choice resulted in one additional 
ime health personnel, in which 
ian di чу (estimated) ofa section other 
It is Ир, Wes Choice was the highest. 
bore _ Bek however, that had the Re- 
p» Кү been equated for length, 
idity Tat significant differences in val- 
Were stir g, footnotes a and b) that 
tà Шо from one Report section 
‘ лег would still have occurred. 
бараке ot the Report sections were made in- 
of obtained vali " apparent from a ve uem 
of validity im BET with the aoe ШШШ 
fications (PQ able 9 that the Personal oe E 
the smallest eee could he expected to snow 
53. On ihe tof Ghat GO ai, hw 
choice (FC) section, 


ever, the ; 
he increase to be expected ranges from 


°з t 
02 te 200, and on the Check List (CL), from 
ето Ий, 


one length of the various Report 
nitude appears to have affected the mag- 
8reater of the validity coefficients, the 
the ee gla scored alternatives in 
Appear ues Choice (FC) section does not 
erally A AN sepan ba for the gen- 
though Kn validity of this section. Al- 
ion m the Check List (CL) was the aed 

. Most affected by the small number 
te oe alternatives, the evidence on 
ke ndicate the 


rel alidity seems to її 
Forced 


ati : qe 
tho; vely higher validity of the 
Ice section. 
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Multiple Correlations 


In order to determine the combination 
of Experimental Efficiency Report sec- 
tions which would, for each occupational 
group, maximally predict the separate 
criteria, multiple correlation coefficients 
based on primary Reports were com- 
puted by the Wherry-Doolittle method 
of test selection (4, Chap. XIV). The in- 
tercorrelations on which the multiple 
correlation work was based are shown in 
Table 10. As can be seen [rom Table 10, 
the intercorrelations among Report sec- 
tions were quite high, particularly in the 
medical and dental groups. In all occupa- 
tional groups, the highest intercorrela- 
tions tended to occur between the Job 
Proficiency (JP) and Personal Qualifica- 
tions (РО) sections, with correlations 
ranging from .66 to .85. The Forced 
Choice (FC) and the Check List (CL) 
ere also highly related, with a 
range in correlations [rom .68 to .82. 

The multiple correlational data are 
able 11, which shows the 
Rs based on selected. predictors, the Rs 
application of the Wherry 
he validity coefficient 


sections W 


presented in T 


obtained by 
shrinkage [ormula, t 
of the best predictor in each occupa- 
tional group, and the beta weights of 


predictors in the order in which the pre- 
dictors were selected. 


As Table 11 indicates, 
nificantly different from zero 
level or below. within each occupational 


group, the increase in validity as а ir 
of using a team of Report sections rather 
than a single predictor may be seen by 
comparing the R with Шет of the ja 
selected predictor. However, since the 
Wherry-Doolittle method 


of test selection 
does not guarantee that the increment 
due to the selection of successi 


уе varia- 
bles significantly increases validity, the 
a "n 
null hypothesis was tested by use of 


the F 
ratio (9: P- 55). Tests 


all Rs were sig- 
at the .o1 


for the significance 


bc 
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TABLE то 
INTERCORRELATIONS AMONG SECTIONS OF THE PRIMARY REPORTS^ 
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cantly different from zero at the .от level or below. 


E 
5 
o 
з 
a 
л 
а 
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of the difference in the R? (or 7?) based 
on the first selected. predictor and the 
R? based on all selected predictors 
showed that, on the Work Performance 
criterion, validity was significantly in- 
creased at the .o level or below in all 
groups except the dental. On the Per- 
sonality criterion, a significant increase 
at the .05 level or below occurred in only 
the largest group, the medical. eH 

As was previously noted in the dizour 
sion of the validity coefficients in Table 
3, the data in Table 11 indicate that ma 
validities obtained against the Work 
Performance criterion were higher than 
those on Personality, and that the phys 
cians group was the one in which predic 
tion was best. 

The relative effectiveness of the 2 
Report sections аз measures of Re 
formance within each occupatio? 
group is evident [rom the results of t 
test selection. The predictors 1 by 
11 are those which, as determined o 
the Wherry-Doolittle method, ашк. 
to produce maximum multiple corre 


р А . in which 
с я order in V 
tions. Inspection of the aum- 


four 


Table 


were selected shows that, cons 
both criteria, the Forced Choice lev 
Section was a selected predictor іп 6 
of the twelve groups. Further, t 
Choice section was the first sel 
dictor in nine of the groups: ag the 
sonal Qualifications (PQ) section Report 
next most frequently selected office! 
section occurring in eight ot iur gp 
groups, while the Job Proficiency each 
section and the Check List 
occurred in three groups. In 8 
Forced Choice section in COT et 
with one of the rating scale Е" 
usually Personal Qualification rio” 
to produce the maximum Gor 

with the criteria. 
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Since the multiple correlational work 
was based on validity coefficients ob- 
tained for item-analysis samples, the 
findings concerning the relative effec- 
tiveness of the various Report sections 
and the sizes of the multiple correlation 
coefficients require verification on inde- 
pendent samples. Evidence from the 
“cross” scoring keys, however, indicated 
that little decrease is to be expected in 
cross validation of the Forced Choice 
section of the Report. Further, cross 
validation of the Personal Qualifications 
and Job Proficiency scales was not 
deemed necessary since matched samples 
had produced highly similar scoring 
keys. In view of these considerations, it 
is likely that the validity coefficients will 


not show a marked decrease in subsc- 
quent samples. 


Validity of the Officer's Progress Report 


Validity coefficients based on the Rat- 
ing Scales (RS) and the Narrative Com- 
ments (NC) sections of the Officer's 
Progress Report and on sections of the 
Experimental Report completed by the 
primary supervisors are shown in Table 
12. Considerable attrition in the number 
of Experimental Reports Occurred as a 


result of using only those ratees on whom 


both Reports were available. The 
median correlations for the Experi- 
mental Report shown in Table 12, how- 
ever, are about the same size as the corre- 
sponding median validities in Table 3. 


Tests of the significance of the difference їп 
the validity coefficients in Table 12 from one 
Report section to another revealed that coeffi- 
cients in 20 per cent (30 out of a possible 150) of 
the comparisons differed Significantly at the 95 
level or below. The specific comparisons which 
produced significant differences are shown 
Table 13. The percentage of significant com 
sons was less than occurred in the tests of d 
ences on the Experimental Report (see Tabl 
this was probably due to the smaller numb, 
cases available on the combined Reports, 


As was previously found, the Forced Choice 


in 
pari- 
iffer- 
€ 4); 
er of 
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TABLE 
XPERIMENTAL REPORT AND THE OFFICER'S PROGRESS REPORT 


FFICIENTS FOR THE E 


VALIDITY COE 


Work Performance Criterion 


Personality Criterion 


Progress 
Report 


1 Report 
eports) 


rimary R 


Experi 
(P 


Report 
Reports) 


(Primary I 


Experimental Ё 


Occupational 
Group 


© 
= 


ЕС 


о 0 I~ 
Qe in 


Nurses rated by phys. 


Nurses rated by nurses 
Dentists 


P.h personnel 
Res. personnel 


Physicians 


+33 


-49 


о 


Median r 


gnificance. 


4 т does not reach the .o5 level of si: 


perimental and Progress Reports were available. 


zero at the .от level or below. 


ked are significantly different from 


° ris significantly different {rom zero at the .o¢ level. 


* Based on the number of officers on whom both Ex 
У АМ rs not mar! 


We MEE LLL ERE S ERR 


METHODS FOR EVALUATING PROFESSIONAL HEALTH PERSONNEL 21 


© TABLE 13 
Омр. : 
MEMO D or SECTIONS OF THE EXPERI- 
Ren EPORT AND THE OFFICER'S PROGRESS 
SPORT IN WHICH VALIDITY COEF 
DIFFERED SIGNIFICANTLY 


| Work Per- | Personalit 
| formance een 
| Criterion 
Occupational 
Group Report Report 
Sections Sections 
Compared^ Compared* 


FC vs. CL 


NS 
Physicians 
РО vs. CL* 


I ublic health per- FC vs. ЈР» 
аша FC vs. РО» 
FC vs. RS? 

FC vs. NC? 


Rese; 
esearch personnel ded. D 
ЕС уз. P 
FC vs. RS 
FC vs. NC" 
PQ vs. ЈЕ 


N ь 
Urses rated by ч FC vs. NC* 


nurses 


Nurs " E 
pon rated by 
бра RS v 

РО v 

RS vs. NC 


^ The - 
validité Report section on which the higher 
Май уште is listed first. 
differ ae ity coefficients for the sectio 

° aa candy at the .or level or be 
differ si ity coefficients for the sections comp: 
ignificantly at the .o5 level. 


ns compared 
low. 
ured 


of 30) of the 
ts than any 
was а Va” 
section 


(EC ur 
ipa on Lea more (18 out 
Other Re y ıigher validity cocfficien 
lidity pow section. In no instance 
Significantly loy on the Forced Choice 
n ИШ lower than that of another m. 
le Жейин, of comparisons in which са 
Validity si ning five Report sections produced 
‘ange Баппу higher than another section 
om none on the Check List to five on 


t 

И Заоа] Qualifications section. 

oe the results of the eee a 

validity е tests and from the me p 

Note "s coefficients, it is interesting. 
hat the two sections of the Progress 

“port, Rating Scales (RS) and Narra- 


tive Comments (NC), produced validities 
that compare favorably with all sections 
of the Experimental Report except the 
Forced Choice. 

The data on the Officer's Progress Re- 
port again suggest the relative superi- 
ority of the forced choice type of evalu- 
ation as compared with more conven- 
tional rating methods. However, since 
the Progress Report was completed 
under operational rather than experi- 
mental conditions, no attempt will be 
made to compare the two Reports by use 
of multiple correlational techniques. It 
is anticipated that in a later study, it 
will be possible to collect data on the 
Progress Report along with cross-valida- 
tional data for the Experimental Report 


so that a more intensive comparison of 


the two Reports can be made. 


SUMMARY 
ared the relative 


This study has comp 
technique 


efficacy of the forced choice 
with other more conventional evalua- 
as measures of the per- 


formance of professional health person- 
nel working as commissioned officers in 
the United States Public Health Service. 

Four sections of an Experimental 
Efficiency Report were studied: (а) 50 
Forced Choice tetrads adapted from 
items developed by the Department of 
the Army; (0) a ten-point scale for rat 
ee's Job Proficiency 1m his pri- 
(с) сїрїї ten-point 
ation of Personal 


the evalu 
d (d) a twenty-two-item 


tion. methods 


ing a rat 


mary job function; 


scales for 
Qualification 
Check List d 


s; an 
eveloped from comments 
ring in the Officer’s Progress Re- 
port, the efficiency report in operational 
e in the Service. In addition, two sec- 
tions from the Officer's Progress Report 
able for comparison with those 


;erimental Report: (a) eleven 
Scales for evaluating 


a ppea 
us 
were avail 


in the Ex} 
five-point Rating 
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various aspects of performance in the 
Public Health Service; and (b) Narra- 
tive Comments coded and scored by a 
method previously developed. 

The criteria of Service performance 
were twenty-point graphic rating scales 
for the evaluation of Work Performance 
and Personality. А ratee's criterion score 
was the average of the ratings given him 
by his work associates on each criterion. 
The results of the study have shown that: 

1. The Forced Choice section of the 
Experimental Report was highly effec- 
tive for evaluating the performance of 
professional personnel commissioned in 
the Public Health Service. 


Of 24 validity coefficients based on scoring 
keys developed by selecting the best tetrads from 
those which had the same empirically deter- 
mined scoring weights in independent matched 
samples, 41.7 per cent were 62 or higher. АП 
except one of the coefficients were significant at 
the „от level or below; this one was significant 
at the .o5 level (Table 2, “combined scoring"). 

Only 12.5 per cent of 24 validity coefficients 
based on item-analysis samples showed a sig- 
nificant decrease at the :05 level or below in 
cross validation (Table 2 


2, comparison of "self" 
and "cross" scoring). 
2. The validity of the Forced Choice 


section was generally higher than that of 
the other Report sections studied. 


Out of 36 significant differences (05 level or 
below) obtained in comparisons of the v. 


= alidity 
of the Experimental Report sections, 2 (75 per 
cent) involved higher validities on the Forced 


Choice tetrads, while only one involv 
coefficient on this section (Table 4). 
The Forced Choice section contained a 


ed a lower 


> greater 
number of scored alternatives than the other 
sections of the Experimental Report; estimates 


of validity based on theoretically making each 
section infinitely long, however, seemed to indi- 
cate that the length of the Forced Choice sec- 
tion was not primarily responsible for j 
erally higher validity (Table 9). 

Out of 12 multiple correlation coefficients 
computed on the Experimental Report by the 
Wherry-Doolittle method of test selection, 11 
involved the Forced Choice section as a selected 
predictor; in nine of the 11, this section was the 
first selected predictor (Table 11). 

Comparisons of the validities of sections of 


ts gen- 
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both the Experimental Report and the Officer's 
Progress Report revealed 30 significant differ- 
ences; 18 (60 per cent) involved higher validi- 
ties оп the Forced Choice tetrads while none 
involved a lower coefficient on this section 
(Table 13). 

3. OF six occupational groups for 
which separate scoring keys were de- 
veloped for the Experimental Report, 
the largest group, that of hospital physt- 

К i; . H Н st 
cians, was the one in which the ре 
validity coefficients generally осситтес. 
than 

the 
ет- 


The occupational groups, other. 
physicians, which were involved in 
study were dentists, public health p 
sonnel, research personnel, and muse 
rated by two different criterion rate! 
groups, physicians and nurses. 


Of 44 significant differences (05 level ог her 
low) obtained in comparisons of validity © Ê 
cients from one occupational group 10 915 33 
on sections of the Ехрегитє! | вера the 
(75 per cent) involved higher coctlicients 1 
medical group (Table 5). 

Multiple correlations (R) for the Doolittle 
Report computed by the Wherry Ду 
method were, for the medical group, 68 a Per- 
against the Work Performance and giants 
sonality criteria, respectively. Both a : ant 
were significant at the .o1 level or be s level 
both represented a significant increase b on th 
or below) in validity over that obtaincc ple cor” 
second-best single Report section. MPD al 
relations in the public health and the res 


Experimental 


ranging 20" 
groups were also relatively high, ranging 
‘57 to .67 on the two criteria (Table п). 
oenerally 


4. Validity coefficients were & rather 
higher when Work Performance !* cri- 
than Personality was used as thé 
terion. 


tal Repo 


On all sections of the Experimen ities We"? 
€ js rf 


except the Forced Choice, higher vali 


jon 

r ә crite! 
obtained with the Work реколтата готсей 
than with the Personality criterion. higher 


Choice validities were not consistently 
for either criterion (Table 8). . 

Within each officer group, a hig 
correlation coefficient was obtained for ‘or 
perimental Report when Work Ре 180 
Was used as the criterion than when Ре i 
was used (Table 11). Exper. 

Considering sections from both the nigh 
mental and the Progress Reports, 


iple 
her тир 
r the “a 
manc 
T" 
nali 
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validiti TES s 
2 а occurred in all but one instance when 

ork Performance rather than Personality was 
used as the criterion (Table 12). 


5. Experimental Reports completed 
by a group of supervisors independent 
of and at a higher administrative level 
than those completing the Reports used 
in item analysis produced validities that 
compared favorably with the validities of 
the item-analysis Reports. 


oe of validity from one level 
validity visor to another, 21 involved higher 
mw coefficients on item-analysis Reports, 
Л AE rene higher validities оп Reports 
visors Ti by an independent group of super- 
ters de median difference in validity coef 
analysis 3. those comparisons in which item- 
Was + ; eports yielded the higher validities 
ens and in those in which the independent 

s gave higher coefficients, 08 (Table 3). 


ü 6. Validity coefficients based on sec 
dae of the Experimental Report did not 
5 » з 3 

low а consistent trend аз а function 


of grade level. 


la possible comparisons of validity coef 
per xs from one grade to another, only 16 (13.1 
pis d yielded differences significant at the 
ap or below. Validity coefficients for 
based € grades were also compared with those 
grades on all grades. The effect of combining 
validit appeared: to be the masking of the higher 
only 2 obtained in certain specific grades; 1n 
Higher d instance was a combined grade validity 
Separa than any of the coefficients for the 
Parate grades (Table 6). 
E P The sections of the Experimental 
ficiency Report exhibited satisfactory 
reliabilities. 


ij rma prow estimates of reliability for 
97 MOT Report. sections ranged from 78 А 
83, res ian reliabilities (ra) were .95, 99% ar 
the ыш a for the Personal Qualifications, 
dme ced Choice, and the Check List ое 
Coefficient Б possible to compute а split-halé 
it consi t for the Job Proficiency section si : 
s CE of a single rating scale (Table 8 
eports measure of rater agreement, SCOTES Er 
at diffe completed by two groups of supervise 
ver жш; administrative levels were corre! oom 
Sets of alf of the correlations between the tW 
Reports were .55 ОГ higher (Table 7): 


8. The Rating Scales and Narrative 
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Comments sections of the Officer’s 
Progress Report appeared to be about as 
adequate measures of performance as 
sections of the Experimental Report 
other than the Forced Choice. 


Median validity coefficients for the Progress 
Report compared favorably with those for sec- 
tions of the Experimental Report other than 
the Forced Choice. Data on the two Reports, 
however, were collected under different condi- 
tions so that comparative results are viewed as 
tentative (Table 12). 

The significance of the difference was tested 
in the validity coeficients obtained for the vari- 
ous sections of both Reports. Significantly higher 
(.05 level or below) validities occurred on each 
section of the Progress Report about as fre- 
quently as on the Experimental Report sections 
other than the Forced Choice (Table 13). 


9. Multiple correlations computed оп 
the Experimental Report indicated that 
prediction was in some instances, but not 
in others, increased by the use of more 


than one Report section. 


АП multiple correlations were significantly 
different from zero at the .o1 level or below. 
Five of the six correlations based on the Work 
Performance criterion represented a significant 
level or below) in validity over 
e best single Report section 
for each officer group. Only one of those based 
on the Personality criterion, however, showed 
such a significant increase (Table 11). 


ation of sections of the 


10. The combin 
Experimental Report which produced 
the maximum correlation with the cri- 


as determined by the Wherry- 
le method, differed for each of the 
tudied, but tended to in- 
d Choice in combination 
ating scale sections, 
alifications. 

2 i relations com uted, six 
odes Eu Choice and peal Quali- 


fications sections as the only selected predictors, 


i ections in combi- 
E hree involved these two s n 
sare ith a third . In one multiple 


he Forced Choice section was se- 


increase (.05 
that obtained on th 


teria, 
Doolitt 
officer groups $ 
clude the Force 
with one of the г 


usually Personal Qu 


, and cac 
redictor selected in the two Те! 


correlations (Table 11). 
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IMPLICATIONS OF THE FINDINGS 


Evaluation. of the performance of 
highly trained professional personnel 
poses a difficult measurement problem. 
The complexities of the work require- 
ments for such personnel make adequate, 
objective criteria of professional compe- 
tency difficult to obtain at the present 
time. Any criterion or criteria should 
presumably reflect such personal charac- 
teristics as professional knowledge, judg- 
ment, technical skill, originality, emo- 
tional adjustment, and ability to ad- 
minister programs in a professional spe- 
cialty. While the inadequacies of the 
type of criterion employed in the pres- 
ent work are recognized, practical con- 
siderations necessitated the use of a con- 
ventional work-associates' rating method, 

With the type of item analysis and 
control of experimental variables used 
in this study, it would appear that, 
within the limitations imposed by a rat- 
ing criterion, satisfactory validity and 
reliability of performance evaluation 
methods for professional health person- 
nel can be obtained. Of particular inter- 
est are the results obtained for the forced 
choice tetrads which, under the condi- 
tions of this study, generally produced 
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higher validity coefficients than ME 
methods of assessing or reporting C" 
ciency. Since methods of 
efficiency reporting have widespread 
usage, it may also be of general тен 
that these methods produced satisfactory 
validity as measures of professional per 
formance. 

The findings appear to be app! ; 
to other organizations employing ш, 
cal, scientific, and other health personne 
similar to those employed by the рр 
Health Service. With regard to the forcec 
choice items, it may be recalled that п 
items used here, although scored by key 
developed from item analysis ol oi pni 
mental Efficiency Reports completed a 
Public Health Service personnel, um 
developed in another organization on a 
employee population quite pared 
from that of the Public Health Servi 
From the evidence concerning үа 
of the tetrads in the variety of S 
activities in the Public Health as 
(medical care, research, and p" » 
health), it may be inferred that the weit 
content and the technique are e at 
as to be relevant in a number of differe" 
employment situations. 
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APPENDIX 


1 | 
| А. SAMPLES OF ITEMS FROM SECTIONS OF THE 
EXPERIMENTAL EFFICIENCY REPORT 


Section I. Forced Ghoice = 
Most Least 


Directions " 

{Ый se for completing: From cach of the 

ag sets of four w rds +4 

the o words or phrases, mark A. С F 

ne wor г а ; 1 А А. Cannot assume гез S 

“йо s word or phrase in cach set which is bility 'esponst 
descriptive" and the onc which is "least 


descriptive" 
live > 3 " ; z 
1 of the ollicer you are rating. B. Knows how and when to 


delegate authority 


| = С, Offers suggestions 


D. Too easily changes his 
ideas 


А. 2 
5 go-getter who always 
oes a good job ` 1 і i 


Most | Least 


В. 


n under all circum- 
А \ 1 
A. Modest and reserved | 


©. „йе 
Dosen t listen to sugges- 
TES = 1 1 
bee B. Doesn't have the drive or 
+ Drives instead of leads \ I force he should | 
C. Antisocial | 
D. Respected by all fellow 
officers | 


Section H. Job Proficiency 


are evaluating. Rate th b | 
in this function by marking а positio 


ten-point scale. 


he officer's job proficiency 


Directi 
ections ; " A 
ons for completing: From the Service ү 
n on 1 


Unctic 

k ms lis 

sider oe below, select the one you con- 
be the primary job of the officer you 


|2 0 = — س‎ = 
Peration i 5 a a 
ation in a technical or specta 


2. Care Н d Public Health program 
3 of patients or furnishing servi i 


5 to patients 
b ram at any level 


i H С 
4 ministration of a clinical or medical care pros 
PE y performing research work 
ы RATIN FIC] g 6 8 о 19 
RATING OFFICER 1 2 
CI Number of Function каии BEN 


І, personal 
rate the officer оп each 


Section П 


Directi aus 
of ea completing: Ву marking 2 position or 
owing personal qualifications. ج ي‎ 
a 
=== 8 то 
| ЖЕКЕ; | 


eri 
а degree to which he is able to discrimina 
А acts to arrive at logical conclusions: 
The de ج‎ ۴ т à 4 5 7 
Suse e to which his appearance and behavior "HD 1 1 i 
Th people to react favorably- MM LL war == —=== 
ed 4 с i 
HEN su ZB {т fut 


; "UL 
T 


owhich he isle T n 
eines E Hb RGE UUAN 
"ry & firmness (ШИ 
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Section IV. Check List statement applies to the officer under considera- 
tion. If a statement does apply, mark space one 
Directions for completing: From the follow- (1); if it does not, mark space two (2). 
ing statements, determine whether or not each 


Applies Does not Apply 


. . T ion 
== = This officer has a broad and detailed knowledge of his professi 
I 2 | 
= = This officer's usefulness is limited to a narrow field 
: s i j i 1 organizing his 
= = This officer does an excellent job of planning and organizi 


work 


B. SAMPLES OF ITEMS FROM SECTIONS OF THE 
OFFICER'S PROGRESS REPORT 


Rating Scales 


Ба E — ڪ‎ 
Indi i Я " Unsatis- - Very Excellent 
ndicate rating by check mark {йб Fair Good ced | 
Judgment — 
General professional knowledge en سے‎ 
Proficiency in assigned duties ے‎ =. | === — 
Tact a 
General fitness for th i سے‎ 
I e service = —— 
Questions Eliciting Narrative Comments 
Are you satisfied to have this officer? Yes O No 0 Give reasons naa шашай i aa 
а MM. 
What are your recommendations for this officer's ітргоуетепір „мураны крк m a Ж." 
ыыр сен е ЖАНАА. 
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I. INTRODUCTION 


Win one interviews large numbers 
"M = bene colege freshmen with a 
Bonos nda Des needs, one finds 
б ет, ar bul all too easy—to divide 
Vise eae Sed. eni On the one hand, 
BE ave lose who seem primarily to 
ханан ар broadening, opening to 
Gs nu en ‚оп the other hand, those 
аєа» ap priamy to need self-disci- 

ganization or integration. The 
a 


first oy 
ا ا‎ seem to have arrived at 
haps оче Ла [көңү which is per- 
Wonders енй and brittle, and one 
Without E € much change can occur 
shattering: a iences that might be 
attained hori pron group seem to have 
quite emis a reedom than they are 
question as te i there. being a serious 
Sion of im hei ik hether so much expres- 
trol by K Ki is under adequate con- 
possibility Pii to say nothing of the 
Jects are “ зоте ol these latter sub- 
somewhat Rise ridden.” To speak 
Strong, wher те strictly, the impression is 
that a qi 1 one studies these freshmen, 
Portance туен of considerable im- 
extends b hag very broad one which 
anism, е authori- 
one extreme ssiveness, and passivity 
bue ilit, to impulsiveness, irre- 
OUsness adven- 


рге; 
assertiveness, and 
ay at the other. 
“ Tesear И n 
chan ch program that is [ocused 
anges j к à 
a S in personality during the 
ars "^ 
, we included measures of 


trends that s re or les 
this broad Шоп p n ed 
compulsiveness, сото] exi juo Lise 
have used the Social бабам но 
("Re") scale of the Californi we 
datas Á а ша Psycho- 
сы P EM (12), the F scale (1). 
and Pt scales of the Minne- 
sota Multiphasic Personality Inventory 
("ММРІ") (13), and the Fundamentalist 
Belief items of Barron's Inventory of 
Personal Philosophy (2); and with the 
accent on impulsiveness, expression, and 
the like, we have used the Delinquency 
("De") scale of the California Psycho- 
logical Inventory (“ОРГ”), the D and 
Ma scales of the MMPI, Maslow's scale 
ominance Feeling in 
nd the Disbelief items 
al Philosophy. 


for measuring D 
Women (19, 20), а 
of the Inventory of Person 
the scales within each 
ps correlate positively with 
ly with the scales of the 
les of college women. 
es have made sense 
predict external criteria, Thus. 
withdraw from college in the 
are higher on De and lower on 
tudents who remain, And seniors 
n freshmen on ре, Dominance 
lower than freshmen on Re, 
clicf. How shall we define 
ales representing the two 
dimension have jn com- 
› us that the compulsive- 
has been better understood 
Our own "idcology-free" 
horitarianism in per- 
er the ground in this 


As was to be expected, 


of these two grou 
ier, and negative! 
in our samp! 
this, these scal 


each otl 
other group, 
More than 

when used to 
students who 
freshman year 
Re than are $ 
are higher thar 
Fecling. Disbelief, 
Е; Fundamentalist 
t is that the 5С 


what i 
ur broad 


extremes of о! 
тот 
controlled CX 
and better measured. 
scale for measuring aul 
з! pa po ae m from compulsive- 


ji ‚‚ “їтесйо 
area. But obviously, free 
pas the F scale” covers an enor- 


ness” ОГ “Jowness ОП 
mous amount of territory that sc 
1 PE pe ess Res 
гал! E 
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have been very fully explored. Measures of De- 
linquency, of Depression, of Mania seem rather 
inappropriate in a study of college students. 
Although these scales have been of value, their 
distributions in our samples have been markedly 
skewed, with consequent loss of discrimination. 
Dominance Feeling and Disbelief seem aimed in 
the right direction, but they could hardly be 
expected to cover all the ground that would ap- 
pear to lie there, d 


It seemed to us that there was need for 
an instrument that would measure a 
general readiness to express, to seek 
gratification of, impulses, in overt action 
or in conscious feeling and attitude. This 
readiness, though not so extreme as to 
produce high scores on the Delinquency 
or Mania scales, might nevertheless dis- 
criminate those subjects who, as noted 
above, seem to be in need of self-disci- 
pline and integration; and it might 
prove useful in further research. 


П. THE IMPULSE-EXPRESSION. SCALE 


The first step in constructing the 
impulse-expression scale (hereinafter 
called for convenience the J scale) was 
to choose, from among the 677 true-false 
items in the test battery being used, those 
items which seemed most expressive of 
a readiness to gratify impulses directly in 
action. This procedure yielded 79 items 
that seemed clearly to express such tend- 
encies as aggressiveness, rebelliousness, 
defiance, competitiveness, restlessness, ex- 
citability, adventurousness, unconven- 
tionality, sensuality, exhibitionism, toler- 
ance, permissiveness, flexibility. The 
7g-item test was first scored for 237 
seniors, and the correlations of each of 
the 677 items with this test score were 
obtained by a short method (25). The 
123 items having correlations at the .oo1 
significance level with the 7g-item test, 
and which also had variances exceeding 
.08, were selected to comprise a second 
approximation to the final test; 77 of the 


TABLE 1 


Means, STANDARD I 
RELIABILITY С‹ 


Axp КК-2! 
IR THE J 


SCALE FoR THREE s or WOMEN 
Group N X 5 | Tu 
Seniors 164 48.60 14.24 p 
Freshmen 220 41.43 Das Sh bos 
ae 5 8.06 12.90 а: 
Alumnae 50 38.00 a 


79 items from the initial criterion к 
had positive correlations, although on : 
49 of these (which also had sufficien 
variance) reached the .oo1 level of 88" 
nificance. " 

The 123-item test was next scored for 
: yrs and 220 
correlations 


two new samples, 164 senic 
freshmen, and  item-test 
were obtained for each. Differences be 
tween the two sets of item иеше 
for these two samples were small ii 
could easily be attributed to en 
Nevertheless, the 7 items for which Me 
discrepancies were greatest were wee as 
to be discarded subsequently, son Pn 
an added precaution to ensure Y for 
test should measure the same thing 
both freshmen and seniors; all retal! <- 
items correlated at the .oo1 level № Ld 
nificance in the combined sample. 23° 
mean item-test correlation for es pet 
item test for these two new samples "^ 
.32. 
Next the 123-item test was SCO" 
50 alumnae of the same соПев‹ ics 
ages ranged from 38 to 46. Ышш 
this sample, and for the 164 511 


szem їй. D 
the зоо freshmen, are given 1n 
cou 


ained 


ors anc 
able ! 
id not 


it had been used for selecting t ces 
The two largest mean aie The 
Table 1 are both easily significa ail 
fact that seniors possess more e 
Expression than either freshm® 
middle-aged women will be us¢ 
an argument for the validity of the 
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The difference between the freshmen 
and alumnz means is not significant. 
The obtained distributions were quite 
symmetrical, but appeared to be slightly 
platykurtic, The reliability coefficients 
of Table 1 are lower-bound values, and 
KR-20 would have given slightly higher 
(but still lower-bound) values; KR-20 
Was computed for seniors only and found 
to be .886. Considering the diversity of 
the samples, the reliability appeared to 
be high enough, but the final revision of 
the test probably increased it slightly. 
Statisticians differ somewhat in recom- 
mending how best to use internal con- 
sistency as а criterion for test construc- 
tion, Ferguson (7) warns against using 
it as the only criterion. Cronbach and 
others (5, 16, 17) have discussed proper- 
ties of the most popular measures of in- 
senna] consistency, the Kuder-Richard- 
n _ Coefficients (14), which provide 
"ntuitively good indices of the unitary 
ме of what is measured by a test. 
е factor tests measure single traits 
aene do tests which have high 
Bo coefficients, but may 
ореп oi es be multifactorial, is still an 
Mo EM there appears to be no 
ogical d reason why a single psycho- 
should тай, as it occurs in personality, 
Other hen be multifactorial. On pa 
item Moni increasing homogeneity y 
às а жоо, еуеп though insufficient 
lest, ан cupa ensure a single factor 
high б t will produce a test ИШ 
he ep actor saturations for jost i 
esit t According to the criterion O 
sented "cnm, the items to be me 
lat we pes comprise a good uides 
рле have chosen to call Imputs 
ion, 

е test consisted 
items referred 
new 


The f T 

А ri final revision of th 
‘Scarding the seven 
reyi i ‘ 
Previously and adding eight 


items which had a mean item-test corre- 
lation of .43 for the Senior sample. The 
new items were obtained from a Weltan- 
schauung inventory constructed іпае- 
pendently by Richard Jung. They were 
added after a study of item clusters 
showed that they would contribute sub- 
stantially to test content. 

The 124 items comprising the final 
J scale are given below, arranged into 
clusters according to content. The 
clusters represent agreement among the 
authors on certain general tendencies 
tapped by the scale, and they are pre- 
sented as an aid to its understanding. 
The items could, of course, be classified 
in several! different ways. Murray's varia- 
bles of personality (22) seem particularly 
well suited to the task. Not only is it 
easy to regard each item as primarily an 
expression of some one of Murray's needs 
or traits, but his scheme offers terms for 


stating the positive side of the case, that 
hat the subject is 


is, for characterizing W 
seeking or exhibiting, so that we are not 
reduced to indicating merely that there 


is "absence o[—" or “freedom from—.” 


Moreover, the scheme offers more or less 
neutral rubrics for phenomena which 
would otherwise have to be described in 
the language of psychopathology. 
previous work has shown that virtually 
all the needs and traits necessary to à 
classification of the present items tend to 
be positively correlated. Sanford (24), 
using data derived from ratings of the 
overt behavior of school children, per- 
formed à correlation analysis of these 
needs and traits, and educed several clus- 
s which can be of service here. We 
y group items that appear 
to belong together, on the basis of their 
aning, but we may be guided 
hat has actually co- 


ter: 
may not onl 


by knowledge 
hered in previous work. 
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In the following, under each major 
heading, we list the needs and traits 
which have been shown to be intercorre- 
lated and which are expressed in the 
items. “Т” or "F" in parentheses after 
each item indicates the direction of re- 
sponse, whether "true" or “false,” of 
subjects scoring high on the scale. 


A. Ascendance (n Dominance, п Aggres- 
sion, n Autonomy, п Acquisition, n 
Recognition, n Defendance) 


Sanford (24) thought that the major 
theme of this cluster was “aggressive sell- 
seeking,” and that a basic concern was 
with “raising the status of the self,” the 
subjects who exhibited the pattern being 
“not concerned with maintaining socially 
approved standards of conduct.” 


n Dominance, n Recognition, n Aggression 
When I work on a committee I like to take 
charge of things. (Г) 
I tend to ignore the feelings of others when 
accomplishing some end that is very important 
to me. (T) 
I would be uncomfortable if I accidentally went 
to a formal party in street clothes, (F) 
When someone talks against certain groups or 
nationalities, I always speak up against such 
talk even though it makes me unpopular, (Г) 
I dominate many of the men I know of about 
my own age. (T) 
Many of my friends would probably be con- 
sidered unconventional by other people. (T) 
would be uncomfortable in anything other 
than fairly conventional dress. (F) 
have often met people who were supposed to 
be experts who were no better than I. (Т) 
would like to hunt lions in Africa. (T) 
dislike women who disregard the usual social 
or moral conventions, (F) 


= 


= 


-= 


n Autonomy, n Aggression, n Acquisition 
must admit I find it hard to work under strict 
rules and regulations. (T) 

I have often either broken rules (school, club, 

etc.) or inwardly rebelled against them. (Т) 
I have often gone against my parents’ wishes. (T) 
I have sometimes wanted to run away from 

home. (T) 

I have the wanderlust and am never happy 

unless І ат roaming or traveling about. (T) 
As a youngster in school I used to give the 

teachers lots of trouble. (T) 
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i 3 rincipal 
In school I was sometimes sent to the princi} 


for cutting up. (T) 

At times I have been 
cleverness of a 
would get by with it. (1) E 

Port — 45 {СГ 

During one period when I was a young 

а i озер TES 
engaged in petty thievery. ( ) me 
If I could get into a movie without paying | 4б 
ld probably 


so entertained by the 


1c 
crook that I have hoped 1 


be sure I was not seen, I wou 
it. (T) 
At times I have a strong urge 
harmful or shocking. (1) 
Once in a while I feel hate toward те 
my family whom 1 usually love. (1) 


to do something 


embers of 


Н " retentions 
n Defendance—with n Blamescape, n Re 


апі Projectivity 

I feel that it is certainly best to keep " 
shut when Em in trouble. (T). 

If several people find themselves i 
best thing for them to do is to 
story and stick to it. (T) 

1 think nearly anyone would tc 
out of trouble. (T) 

It is a good thing to know people i» 
places so you get traffic tags and suc 
taken care of. (T) | ۳ 

It is a good rule to accept nothing as € 
proved, (T) | 

I go out of my way to meet trouble 1 
try to escape it. (T) TE OUI 

I would do almost anything on а {Л Ыы my 

] have very few quarrels with member 
family. (F) 


лу mouth 


1 trouble, the 
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]l a lie to keep 


in the right 
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One of Sanford's clusters Wa pant 
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"Aggressive. Sell-defense"; 15 eo ШР 
tendency was “to protect the sell DY ; or 
doubts 

hat hart 
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ing the offensive, to overcome 
misgivings by adapting a somew , 
and defiant attitude.” It embrace 
the variables listed above pop 
cendance except ? Золин Rejet 
; anc 
tion, n Blamescape, n Retention © pe 
Projectivity. This pattern 5€ 
fairly well expressed in the la 
the preceding 
about n Defendance. It turne 
ever, that Ascendance and ios 
Self-defense were correlated " 
cating a difficulty that will ely; 
much in the present work, name ece” 
whereas it is easy, and no a fensi” 
sary, to separate in theory the Чё 


items—those NA 
d out, ho 
es 
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from the “positive or spontaneous,” it is 
very hard to do so at the level of overt 
expression. Moreover, “where there is a 
tendency to advance the ѕе by energetic 
activity there is likely also to be present 
a tendency to defend it in the same way.” 
It seems just as well here to make one 
grouping of Ascendance and Aggressive 
Self-defense. 


B. Sensation. (n Sex, N Excitance, n 
Cognizance, n Recognition, n Succorance, 
n Exposition, n Blamescape, n De- 
fendance, Projectivity, Impulsivity, Emo- 
tionality, n Change) 


This is another syndrome from Phy- 
sique, Personality and Scholarship (24): 
“а very large constellation which seems 
to denote a general tendency to seek 
for sensation or excitement.” Sensation 
correlated .52 with Ascendance, and to- 
gether they “seem to constitute a very 
general picture of ‘outgoingness’ . . - it 
would appear that in the Ascendance 
syndrome, positive action is the unifying 
tendency, while in the Sensation syn- 
drome it is positive feeling.” The Sensa- 
tion syndrome was thought to embrace 
“a complex of which the main feature is 
the emotional expression of erotic needs.” 


n Sex, n Exposition, n Cognizance 


I like to talk about sex. (T) 

I like to hear risque stories. (T) 

I never attend a sexy show if I can avoid it. (F) 

I have never indulged in any unusual sex prac- 
tices. (F) 

I have been in trouble one or more times be- 
cause of my sex behavior. (T) 

When a man is with a woman he is usually 
thinking about things related to her sex. (T) 

I like to flirt. (T) 

In general 1 prefer the company of men 10 
women (in sports, intellectual activities, hik- 
ing, theatre, conversation, etc.). (T) 

I am embarrassed by dirty stories. (F) 


n Sex, n Excitance—with n Dominance and 
Masochism 


Thave been disappointed in love. (Т) 
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1 like men of whom I am a bit afraid. (T) 

1 rather like justifiable conceit in a man. (Т) 

1 dislike a man who is frequently blunt in his 
speech. (F) 

like men who antagonize me somewhat. (T) 

1 prefer men who are never profane. (F) 
rather like aloofness in a person I have just 
met. (T) 

1 like worldliness in people. (T) 

I enjoy the company of strong-willed people. 


(Т) 


n Excilance 
I enjoy playing cards for money. (T) 
I enjoy betting on horse races. (T) 
When I get bored I like to stir up some excite- 
ment. (T) 
1 have never done anything dangerous for the 
thrill of it. (F) 
Lam fascinated by fire. (T) 
1 like to go to parties and other affairs where 
there is lots of loud fun. (T) 
think I would like to drive a racing car. (T) 
have never done any heavy drinking. (F) 
have used alcohol excessively. (T) 


= 


m 


n Exhibition 


like dramatics. (T) 

would enjoy fame (not mere notoriety). (1) 
would like to be an actor on the stage or in 
the movies, (T) 


n Change-with Disjunctivity 


have periods of such great restlessness that I 
cannot sit long in a chair. (T) 

find it hard to keep my mind on a task or 
job. (T) 

enjoy discarding the old and accepting the 
new. (T) 

have several times had a change of heart about 
my life work. (Т) 

find that a well-ordered mode of life with 
regular hours is congenial to my temperament. 
(F) 

I would rather be a steady and dependable 
worker than a brilliant but unstable one. (F) 
am known as a hard and steady worker. (F) 
] always see to it that my work is carefully 
planned and organized. (F) н 
am very careful about my manner of dress. (F) 


м 


Impulsion 


often act on the spur of the moment without 

stopping to think, (T) 

I often do whatever makes me feel cheerful here 
and now, even at the cost of some distant goal. 
(T) 

Emotionality 
1 get excited very easily. (T) 
Once a week or oftener 1 become very excited. 


(1) 
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I easily become impatient with people. (T) 

I have had periods when I felt so full of pep 
that sleep did not seem necessary for days at a 
time. (T) 

At times I feel like picking a fist fight with 
someone. (T) 

At times I feel like swearing. (T) 

Sometimes I feel like smashing things. (T) 

Often I cannot understand why I'm so cross and 
grouchy. (T) 

I often feel as though I had done something 
wrong or wicked. (T) 

Sometimes I feel that I am about to go to pieces. 
(T) 

I frequently notice my hand shakes when I try 
to do something. (T) 

I work under a great deal of tension. (T) 

Something exciting will almost always pull me 
out of it when I am feeling low. (T) 


One subgroup of items in the present 
set—those which have been labeled n Sex 
and n Excitance—deserve a special word. 
These items are adapted from Maslow's 
scale for Dominance Feeling in Women, 
and our label does not do them justice. 
It is clear that the items express m 
Dominance, n Aggression, and Masoch- 
ism, as well as n Sex and n Excitance. 
As far as we know, however, Maslow's 
scale is the best verbal predictor of sex 
experience on the part of college women 
that has yet been devised. It owes its suc- 
cess to the fact that it comes to grips 
with some of the major psychological 
conditions of such experience—impulse, 
anxiety, effort to suppress anxiety by 
action, and dominant and provocative 
behavior toward men in order to elicit 
from them a sexual approach that is 
sufficiently dominant and aggressive to 
relieve anxiety and to gratify impulse. 

As predictors of overt sexual behavior 
Maslow's items seem not out of place in 
the Sensation grouping. At the same 
time, however, they draw attention to 

the same question that was raised in 
connection with Ascendance, i.e., to what 
extent are we dealing with the straight- 
forward gratification of impulse and to 
what extent with defensive acting out? 


Sanford's "Anxious Emotional E res 
siveness" syndrome, embracing л Exhibi- 
tion, Projectivity, Impulsion, and Emo- 
tionality—which appear in the Sensation 
grouping—and in addition Anxiety and 
Disjunctivity, was said by him to have 
Sensation 
syndrome—the correlation is .73. +++ In 
both syndromes there is outgoing 1 
pulsive and emotionally toned activity. 
but this one is associated with anxiety» 
which gives to the behavior a quality of 
insistence or desperation. It might НА 
that we have here an extraverted reac 


: io as d месі 
tion to anxiety." It is possible to dete’ 
Ё in 


pres- 


“much in common with the 


im- 


a note of insistence or desperation 
many of the present items, but we do по 
have items that express anxiety Or ae 
fensiveness in a pure way. Perhaps this 
is not too much to be regretted, for on 
items would 


the above evidence, such 
the 


correlate highly with the others ol 
group. 


C. Endocathection and Intraception 
Endocathection (as opposed to pee 
cathection), according to Murray (22) 
has to do with the relative importance 
to the subject of “fantasy, en 
imagination or abstract thought,” ae 
“practical, concrete, physical or pom 
action." Intraception "is the dispositio 
to be determined by diffuse 1. 
feelings and inclinations (intangible B - 
jective facts)," as opposed to aes 
tion, "the tendency to be c d 
concrete, clearly observable, physical ie 
ditions (tangible, objective facts). 
two variables are easily confused, 
they tend to be positively un Jess 
(38 in Sanford’s study); ari 
Endocathection may go with em in 
tion, e.g., when a subject is intereste" a 
ideas and theories about substan А 
events, and /ntraception may 89 cadi 
Exocathection, e.g, when a subject t€ 


personal 


Jatec 
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to live imaginatively, to dramatize the 

self, to express his sentiments and be- 

liefs in action. 

In the following items the accent is 
more on Endocathection than on Intra- 
ception, and there are intimations of 
Projectivity and Na rcissism. 

I have had very peculiar and strange experi- 

(T) 

had strange and peculiar thoughts. (T) 

My way of doing things is apt to be misunder- 
stood by others. (T) 

I often get the feeling that I am not really part 
of the group I associate with and that 1 
could separate from it with little. discomfort 
or hardship. (T) 

I brood a great deal. (T) 

L have frequently found myself, when alone, 
pondering such abstract. problems as free will, 
evil, etc, (T) 

I think I take primarily an aesthetic view of ex- 
perience. (T) 

I dream frequently. CF) 

Sometimes without any reason or even when 
things are going wrong, I feel excitedly happy, 
“on top of the world.” (Т) 

A wise person thinks of life as a game; he is both 
in and out of the game and watching and 
wondering at it. (T) 

Some of my friends think that my ideas are im- 
practical if not a bit wild. (T) 

1 have had periods in which I carried on activi- 
tics without knowing later what I had been 
doing. (Т) 

I sometimes keep on at a thing until others lose 
their patience with me. (T) 

My home life wa always happy. (F) 

I have had periods of days, wecks, or months 
when I couldn't take care of things because 
I couldn't get going. (T) 

Most nights I go to sleep without thoughts or 
ideas bothering me. (F) 

I often feel as if things were not real. (T) 


D. Radical Sentiments 

According to Murray (22), “the varia- 
ble stands for the proportion of ex 
pressed sentiments, tastes, and opinions 
that are. (1) novel, original, unique; or 
(2) contrary to those held by the majority 
of respected citizens.” The sentiments of 
religious liberalism or of irreligiousness 
Seem to fall well enough within the scope 
st this definition, though they might 
€asily be taken as a group by themselves. 


The other radical sentiments range over 
a wide area of human affairs. 


The only meaning to his existence is the one 
man gives himself. (T) 

God hears our prayers. (F) 

I go to church almost every week. (F) 

I believe in a life hereafter. (F) 

We cannot know for sure whether or not there 
is a God. (T) 

I pray several times every weck. (F) 

I believe there is a God. (F) 

Organized religion, while sincere and construc- 
tive in its aims, is really an obstacle to human 
progress. (T) 

In religious matters 1 believe I would have to 
be called agnostic. (T) 

Moral codes are relevant only when they fit 
the specific situations; if the situations differ, 
they are merely abstract irrelevancies. (T) 

Man's search for a purpose, ideal, mission, etc., 
is largely a search for a plot or pattern to his 
life story—a story that is basically without 
meaning. (T) 

Children should receive no religious instruction 
of any kind. (T) 

People would be happier if sex experiences be- 
fore marriage were taken for granted in both 
men and women, (T) 

Politically I am probably something of a radi- 
cal. (T) 

1 don't like modern art. (F) 

‘The unfinished and the imperfect often have 
greater appeal for me than the completed and 
the polished. (T) 

I think I am stricter about right and wrong 
than most pcople. (F) 

No man of character would ask his fiancee to 
have sexual intercourse with him before 
marriage. (F) 

I would disapprove of anyone's drinking to the 
point of intoxication at a party. (F) 

It is all right to get around the law if you don’t 
actually break it. (T) j 

I believe women ought to have as much sexual 
freedom as men. (T) 

In illegitimate pregnancies, abortion is in many 
cases the most reasonable alternative. (T) 

1 would be ashamed not to use my privilege of 
voting. (F) 


Although the items seem to fit rather 
well into the above categories, there is, 
of course, the usual difficulty with items 
that seem to express more than one idea 
and, hence, might almost as well be 
placed under some other heading than 
that to which they were assigned. And 
then, as mentioned above, there are 
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TABLE 2 
ConnELATIONS OF J WITH OTHER SCALES E - 
| | Items | Alumnae Freshman 
= No. of ze Sample Sample 
Scales Correlated with J lens | бу а ; ho | 
i me бо 1 
De, Delinquency 54 14 a ae 
Re, Responsibility 41 6 —. ae 
MF II, Passivity | 42 II — > 
MF III, Intraception . 33 5 m 
MF II and MF III (Multiple А) 75 16 P 
M Do, Maslow Dominance 47 17 E | 
123 Ethnocentrism o = uer 
F, Authoritarianism | | e = j | 
Vassar Developmental Scale, De- А si 
velopmental Status 124 37 .68 | E es 
^ A random sample of the freshman sample of ‘Table 1. 
‘ Ч : А . Gn andicate the 
other classification schemes. The items, was designed by Gough "to = pi 
7 e a ; " " esc 
after all, have been taken from existing potentiality for delinquent, troub cbel 
scales-Delinquency, Dominance Feeling, behavior, and the tendency to ^ n 


Responsibility,  Masculinity-Femininity 
and the like—and they might be classi- 
fied on this basis. It would also be possi- 
ble to classify them according to their 
most superficial meaning—rebelliousness, 
criticalness of authority, unconvention- 
ality, and the like. But however fascinat- 
ing the classification exercise might be, 
the most salient fact about the present 
group of items is their statistical homo- 
geneity. The first task is to conceptualize 
what it is that makes for unity in the 
scale. The scheme that we have adopted 
is instrumental to the performance of 
this task. We shall be in a better posi- 
tion to offer a theory, however, after we 
have considered some relations of the J 
scale to other measures. 


III. RELATIONS OF THE J SCALE AND 
OTHER MEASURES 

A. Correlations with Other Scales 

Table 2 presents some correlations of 
J with certain other scales, scales which 
either overlap J or seem to stand in inter- 
esting relationships to it. 

The Delinquency (De) scale from the 
California Psychological Inventory (CPI) 


against authority and convention. d 
has with considerable success ptm 
inmates of correctional institutions [rom 
other subjects. In view of Gough's A 
pression of what his scale measures û he 
of what has been said above ee 
J scale, a substantial correlation Баб 1 
the two instruments is to be е 
It must be noted that 14 of the F oov 
items are actually in the J scale. F 
much the correlations between J by 
other scales of Table 2 are inflated. 1 
е questio! 
lation 
hc 


common items is a bothersom 


(the corre 


“ QU 
common elements 


) : s ansv 
tions) which we do not propose to ? and 
removed, f 


her ег ver 
e. If the overlap were attenu- 


the resulting rs then corrected for 
ation, it is doubtful that they oem 
much, if any, higher than the pod in 
rs of Table 2. Fortunately, nothing - 
our conclusions depends upon ! 
lute size of these correlations; е 15 
number of common items in each ca 
given in Table 2. ale 
The Social Responsibility (кеу 
the CPI successfully identifies 
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who are “seen by others as responsible 
and dependable.” It correlates around 
—.45 with De in large samples of col- 
lege females. The correlations of Re and 
J, —.27 and —.60 (with 6 common items) 
seem to fit in well enough with our con- 
ception of high J as indicating a relative 
lack of concern with social obligations 
and standards of conduct. 

‘The Masculinity-Femininity 
MF II and ME III, require a [ew words 
of explanation. Included among our 677 
true-false items were all the items from 3 
MF scales: the CPI Femininity scale, the 
MMPI Mf scale, and a scale taken from 
the MMPI by Drake (б) which he 
standardized with the use of about 1,800 
college undergraduates. We composed a 
first factor test, f, made up of items from 
these three scales, to which the factor 
gave weights of 5:0, 4-4 and б.о respec- 
tively. These weights were those obtained 
by extracting the first principal com- 
ponent, with unreliability allowed for. 
Since the reliability coefficients of the 
original tests were low (for the Drake 
scale, KR-20 = +35)» it was not surpris- 
ing to find that f had a reliability of 
only .61. Further efforts to maximize 
the reliability of f resulted in a coeffi- 
cient of only .69. The test was therefore 
broken up into three rational clusters as 


scales, 


follows: 

MF I (18 items): Preference for con- 
ventionally feminine roles and in- 
terests. 

ME II (45 items): Lack of aggressive- 
ness, of dominance, of manipula- 
tiveness; docility, modesty, moral 
sensitivity. 

MF III (33 items): Emotionality, fan- 
(азу, introspection, “neurotic 
trends,” and aesthetic interests. 

The reliabilities of these clusters were 


43, .65 and .72, respectively. MF Iis 


represented in the J scale by a few items 
such as “I would like to hunt lions in 
Africa,” but the scale is too short and 
unreliable to deserve serious considera- 
tion. After a study of the item-cluster 
correlations, three items were dropped 
from MF II, raising its reliability to .69. 
Further analysis to lengthen the clusters 
is in progress. 

On the strength of its high correlations 
with MF II, —.62 and —.68 (with 11 
common items), one might be inclined to 
say that the J scale contains a large ele- 
ment of masculinity. But the correlations 
with MF III, -47 and -35 (with 5 common 
items), suggest that J also contains a 
considerable element of femininity. 
There is a comment here on the old 
dilemma of how to define masculinity or 
femininity. We are inclined to doubt the 
fruitfulness of regarding femininity, in 
the sense of what, in attitude and in 
social behavior, differentiates women 
from men, as a single variable or factor. 
MF II and MF III seem to represent two 
quite different ways in which women 
manage the impulse life, by passivity and 
“masochism,” or by elaboration of the 
“inner life.” The correlations with J sug- 
gest that there might be a third way, by 
impulse expression, both in overt be- 
havior and in conscious fantasy and atti- 
tude. 

It is this, it would appear, that is ex- 
pressed in the multiple correlations of 
J with MF II and MF III, .69 and .76 in 
the two samples. It will be interesting to 
know how men score on the J scale. How- 
ever this might turn out, if we should be 
asked to consider how women differ 
psychologically from men, we should be 
inclined to ask in return, “Do you mean 
high-J women or low-J women?" 

The Maslow scale (M Do) deals di- 

rectly with one way in which women 
manage their femininity. It is not con- 
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cerned with sex differences; many of its 
items are not appropriate for men. As 
noted above, this scale taps a complex 
of factors, a syndrome one might say, 
that has much in common with J. The 
correlations, .62 and .52 (with 17 of the 
47 revised Maslow items appearing in the 
J scale), are much as one would expect. 
The absence of correlation between J 
and the E and F scales of The 4 uthori- 
tarian Personality is particularly inter- 
esting. One might suppose that a scale 
that embraced radical sentiments, endo- 
cathection, rebelliousness, and sensation- 
seeking would certainly correlate nega- 
tively with ethnocentrism and authori- 
tarianism. Indeed, опе of the types of 
low scorers on E and F distinguished 
by the authors of The Authoritarian 
Personality was the “impulse ridden.” 
And there has 8rown up a common 
Stereotype of the “low Е personality,” 
one that accents fluidity, lack of inhibi- 
tion, sensuality, Bohemian values, and 
the like. The present results offer little 
support for this notion. Though such 
trends may be found in low Scorers on F, 
they are by no means distinguishing 
features of such subjects. One of the 
reasons why the J scale does not corre- 
late with F, we would Suggest, is that the 
new instrument embodies too much ag. 
gressiveness. In a study of authoritarian- 
ism in relation to Psychopathology (9) 
the present authors showed that im- 
punitiveness is the low Е scorer’s most 
characteristic mode of handling арргеѕ- 
sion. (The Е scale correlated —.41 with 
the MMPI Hy scale minus its somatic 
items.) Note, in this connection, 
absence from the J scale of i 
to do with positive soci 
is no n Affiliation nor n Nurturance, and 
little n Succorance. Unless a Subject has 
some amount of these tendencies, it js 
probably difficult for him to be free of 


the 
tems having 
al feeling: there 


authoritarianism and ethnocentrism. 
The same might be said for Sudan: 
tiousness, The authors of The Author 
tarian Personality have stated that pe 
fortably internalized superego is the i 
guarantee against authoritarianism. | 
this is true, then high scorers on the . 
scale could hardly be expected to mu 
very low in the F scale. Nothing d 
be more out of place in the J scale t a 
expressions of a highly integrated d 
€go. On the other hand, there ae 
pressions of a primitive, — 1 
agency that gives rise to anxiety а! 


need for punishment. minent 

Since anti-intraception is a pram” it 
feature of the authoritarian mnt 
might be supposed that Baia | 
and Zntraception, as expressed in “Chel 
scale, would go with low F gors: some 
this may not be so is food re Intra 
thought about authoritarianism. ne 
ception (being determined by Ende: 
personal feelings and id maior) P. 
cathection (being devoted to S п 
ideas, symbols and artistic produc them 
and Radical Sentiments do not in (hori 
selves stand in opposition to au 
tarianism, 

Тһе Vassar Developmental 
(VDS) is made up of 124 items, 2 xe 
original battery o[ 677, which ps at the 
lerentiated freshmen from senior : cor 
19 1.08 


scale 


om our 
dil- 


:05 level in successive samples. ant 
relations of this scale with J. -5! asisten 
(with $7 common items), аге мар that 
with the fact, presented in Table fate 
Seniors score higher on J ш тё! 
men. The suggestion is that “reins 
like a lifting of repression or : quri” 
impulse occurs characteristically may 

the college years. The alumnæ, imere”! 
recalled, while not significantly 1 wel 
from the freshmen, were definite а hop* 
on J than were the seniors. pe 4 a 
that this reflects, on the part of th 


"em ot 
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TABLE 3 
MMPI MEANS гок GROUPS OF FRESHMEN ARRANGED IN ORDER FROM 
HIGH то Low SCORES ON THE J SCALE 
Group N Li F K Hs Hs? D Hy Pd Pd» 
H | 20 2.9 7.8 13.4 8.4 rer 22.6 22.3 19.2 24.6 
HM 44 3-4 6.7 14.7 6.5 13.0 21.0 22.5 16.7 22.6 
LM 44 6-1 4.2 19.1 4.2 13.8 18.2 21.9 13.2 20.8 
L 20 5-9 4.2 19.2 3.8 12.9 17.6 22.6 13.0 20.7 
Group | N | Mf Pa Pt Pe 5с Se Ma Ma* 
H | 20 34.4 12.0 20.0 34.3 22.0 35.4 20.9 23.6 
HM | 44 34.7 10.2 16.4 31.1 15.2 29.9 19.6 22.5 
LM | 44 35.5 9.0 я]. 26.8 6.7 25.8 13.8 17.6 
L | 20 | 35.6 9.0 6.0 25.2 7.2 2б.4 13.0 16.8 


* Means in these columns have the 


nx, increased control rather than a loss 
of freedom! 

Finally, some nonverbal correlates of 
J are only now being investigated, but 
one finding which seems worth comment- 
ing on has appeared. Subjects in two 
samples (N, = 71, № = 220 freshmen of 
‘Table 1) were presented with a large 
number of figures, or designs, and asked 
simply to indicate whether or not they 
liked each figure. The figures varied ac- 
cording to three dimensions: simplicity- 
complexity, passivity-activity, and mas- 
culinity-femininity. The correlation of J 
with preference for active (as opposed 
to passive) figures was positive in both 
samples. The correlation with preference 
for complexity (or simplicity) or for mas- 
culinity (or femininity) of the figures was 
negligible, but, for example, in the sec- 
ond sample the correlation with a pref- 
erence for active figures was .42. Further 
studies of this difference between high 
and low scorers on J are under way. 


B. Relations Between Impulse Expres- 
sion and the MMPI 

'The relations of J to the MMPI were 
Studied in the sample of 220 freshmen. 
First the subjects were divided into 5 
Broups on the basis of their J-scale 


scores: the зо highest (I); the 44 went 


“R correction.” 


highest (HM), the 92 middle (M), the 44 
who were low middle (LM), and the 20 
lowest (L). This division of a distribu- 
tion is recommended as most efficient by 
Flanagan (8). Then the mean scores on 
the various MMPI scales were calculated 
for each group, except for M. These 
means are shown in Table g. Each row 
of the table may be regarded as a com- 
posite MMPI profile for each group of 
subjects. 

The high and the low J groups dis- 
play rather distinctive MMPI trends. 
Except for the Hysteria scale and, to a 
limited extent, the Hy pochondriasis and 
Masculinity-Femininity scales, the high 
groups score consistently higher than the 
low groups on all scales, with the ex- 
pected reversal of this tendency on the 
L and К scales. Although the L sub- 
jects are not sharply distinguishable from 
the LM's, a definite linear trend is evi- 
dent for the four groups in the scales, 
with the H group scoring highest on the 
clinical scales (lowest on the L and K 
scales), and the HM group scoring inter- 
mediate between the H and LM groups. 

The results shown іп Table $ sug- 
gest a significant relationship between 
J and some MMPI index of repressive- 
expressive tendency. One such index in 
common usage is the [ollowing: + 
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TABLE 4 


DISTRIBUTION ох MMPI SCALES or J SCALE ITEMS 


ммр Scale 


| F K Hs D Hy Pd Mf Pa Pt Se Ma 


No. of J Scale Items | 2 3 6 o 


10 4 8 5 3 4 10 rI 


K + Hy — Pd — Ma. The correlation be- 
tween scores on this index and the J 
score is —.61. The two measures, in other 
words, have approximately a third of 
their variance in common. It is our im- 
pression, based on some consideration 
of individual cases, that a subject who 
is high on the MMPI is very likely to be 
high on J, but that subjects high on J 
may have various types of MMPI pro- 
files, including a generally low one. It 
would be possible, by assigning regres- 
sion weights to the MMPI scales, to ob- 
tain a J x MMPI correlation more ex- 
treme than —.61, but J would still have 
variance that was uniquely its own. ОГ 
the 124 J-scale items, 38 are from the 
MMPI. Table 4 indicates the number of 
J items in each MMPI scale. Since some 
MMPI items are scored on more than 
one scale, the number of items in the 
table exceeds 38. It will be seen that 
the common items are distributed over 
all of the MMPI scales except for the 
Hy pochondriasis scale, the greatest con- 
centration being in the Depression, Psy- 
chopathic Deviate, Schizophrenia, and 
Mania scales, 

To sum up the relations between J 
and the MMPI: higher scores on th 
former tend to go with 
rather than “repression,” 


e 
“expression” 
with a stronger 
inclination toward “psychopathology,” 
and with a relatively stronger tendency 
to employ “psychotic” (the right side of 
the MMPI profile) rather than "neu. 
тойс” mechanisms of defense. It has been 
suggested before that these terms are not 
very appropriate for describing differ- 
ences among college students, Perhaps 


these MMPI terms when applied to pa- 
tients in hospitals and clinics refer to 
more extreme instances of what we have 
called ascendance, aggressive self-defense, 
anxious expressiveness, endocathection, 
and intraception. 


C. The J Scale in Relation to an 
Adjective Check List 


Another source of pertinent data was 
a list of 100 adjectives which the fresh- 
men had marked "true" or “false” to 
indicate whether or not they believed 
them to be descriptive of the “Sell,” the 
"Ideal Woman,” the “Average Woman, 
the “Ideal Man." "These adjectives were 
obtained from one of the earlier forms 
of the Interpersonal Check List of La 
Forge and Suczek (15). They have been 
found useful in previous studies, and 
there is evidence that when they are used 
in such fashion as the above they pos 
Sess retest reliability (4, 15) sufficient Jor 
most purposes, Subjects described Self 
and Ideal Woman on one occasion, the 
two other concepts on another, the same 
list of adjectives being used four ica 

The subjects’ test papers were yan 
into 5 groups according to the size О 
the J score, just as in ‘Table 9. Taking 
Ше 100 adjectives one at a time, the 
number of subjects, in each of the 5 
groups, who answered "true" was ob- 
tained, And this for each of the concepts 
“Self,” “Ideal Woman,” and so on. Table 
5» which gives the results for the adjec 
tive “sarcastic,” is illustrative of the 10° 
tables that were prepared. Reading ни 
опе тау note for each J group, vA 
Highs, the High Middles and so on, ап© 


^ 
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TABLE 5 


FREQUENCIES OF “ TRUE" RESPONSES IN Four Parts or THE J SCALE 
DISTRIBUTION FOR THE ADJECTIVE “SARCASTIC” 


= J Group H HM LM L 2XH 2XL 2XH+HM 2XL+LM 
N 20 44 44 20 

Self 4 35 I1 5 28 10 63 21 

Ideal woman 2 о 1 o 4 o 10 1 

Average woman 5 7 5 3 10 6 17 11 

Ideal man о 8 2 о 18 о 26 2 


for groupings in which the scores for that in interpretations we cannot rely 
the most extreme groups have been very heavily upon the absolute size of 
weighted 2, the number of times "sarcas- these correlations. 

tic" is ascribed to the Self, to the Ideal 'The main conclusion from Table 6 is 
Woman, to the Average Woman and to simply that differences in J scores are 
the Ideal Man. It is not surprising to accompanied by differences in the way 
the self and others are conceived. The 
fact that the correlations are higher for 
“ideal man” and “ideal woman” than 
for “average woman” and "self" may 
indicate that the former are mainly cul- 
tural stereotypes not readily influenced 
by psychodynamic factors. 

The second kind of correlation. ob- 
tained with the present data concerns 
the question of whether, and how, sub- 
jects high and low, respectively, on J 


differ with respect to the relations among 
voman, 


note that the high J subjects think of 
themselves, and of the Ideal Man, as 
sarcastic much more often than do those 
low on J (14 and 5. 9 and о). 

But the content of the adjectives will 
be taken up later. The concern here is 
with the two kinds of correlations which 
were obtained with the use of data from 
the tables illustrated by Table 5. The 
first kind of correlation concerns the 
question of whether or not, or to what 
extent, subjects scoring high and low 
on J differ in the way they conceive of their conceptions of self, of ideal v 
themselves, of the ideal woman, and so and so on. The correlations are shown 
on, Table 6 gives the high J-low J cor- in Table 7, those for the high-J subjects 
relation of the 100 pairs of scores (illus being the upper entries, and those for 
trated in the last two columns of Table the low-J subjects being the lower en- 
5) for each of these concepts. Note that tries, in each row. 

11,000 weighted adjective scores were Viewing the table as a whole one may 
used in the computation of these coeffi- see that subjects high on J show greater 
cients (adjectives used by the M group diflerentiation in their conceptions of 
receive zero weights). There should be self and others than do subjects low on 
no question of their reliability. The 
correlations are, however, inflated by 
variations in the adjective variances, 
that is to say, the totals for all the J 


TABLE 6 
CORRELATIONS (INTERGROUP) OF WEIGHTED 
ADJECTIVE SCORE PAIRS FOR THE 
Grovurs HIGH Ахр Low on J 


groups tend to rise and fall together, as Î тау | seram [^ Ideal 

one considers one adjective alter another, Self Woman | Woman Man 

d Я J = ик =R Е 
€pending on the general popularity of C л "m 88 da 


the a5 х * = 
te adjective in question, This means — 
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INTERCORRELATION (INTRAGROUP) OF W EIGHTED 
ADJECTIVE SCORES FOR THE FOUR Concerts FOR 
THE HIGH J GROUP AND FOR THE 
Low J Group 


Ideal Avg. Ideal 
Woman | Woman | Man 
Self 
High J Group 47 .71 .59 
Low J Group .87 .87 .85 
Ideal Woman 
High J Group -55 .89 
Low J Group .79 -94 
Avg. Woman 
High J Group .58 
Low J Group .8o 


J. This fact is further demonstrated by 
comparing the first principal compo- 
nents (Hotelling factors, extracted after 
placing 1’s in the main diagonal of the 
correlation matrix) for the two matrices, 
the one comprising the high-J and the 
other low-J entries of Table 7. For sub- 
jects high on impulse expression the 
first component absorbs only 72% of 
the variation, for subjects low, 89%. 

Differences in the way subjects high 
and low on J describe the self and ideal- 
ized persons are striking. The ideal-m 
-ideal-woman correlations are large for 
both groups (.89 and .94), showing a 
common tendency to discriminate them 
less than other pairs. The self—ideal- 
woman correlation is much higher (.87) 
for the low-J group than for the high 
group (.47); in fact the latter, unlike the 
former, think they are more like the 
average woman (.71) than like the ideal 
woman (.47). High-J subjects also seem 
to think they are more like the ideal 
man (.59) than the ideal woman (47), a 
tendency which is reversed in the lows 
(.87 compared with .85). 

One interpretation of these findings 
is that subjects high on J are less stable- 
or should’ one say less rigid?-in their 


an 
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identities than are the low subjects. B 
may also be commented. that relatively 
low self-esteem, perhaps even $e 
and self-indulgence. It may well be 4 
the former is one of the forces behine 
the latter, Or perhaps the two are mu 
tually influential. г the 
Turning now to the content d m 
adjectives, to qualitative differences a 
tween subjects high and low on J M 
note considerable difference between : E 
two groups in types of adjectives pin 
as Table 8 indicates, Table 8 Lc 
for the conceptions of "Sell, par 
Woman,” “Average Woman,” and Ic и 
Man" those adjectives which шек " 
ate a group of high-J subjects (N — 64) 
from a group of low-] subjects GN = all 
at the .05 level of significance. ып at 
adjectives were marked either true dt, 
false by all subjects for each ipe 
interchanging the headings “True 4 " 
"False" in Table 8 will give the d 
frequent direction of response fot ‘oh 
scorers, as contrasted with that of hig 
scorers. [- 
As can be seen from Table 8, the pe 
concept of the high group amp 
qualities of aggression, suspicious! ир, 
and stubbornness; that of the low 8! е 
cooperativeness, affiliation, and uin 
ance. The concept of the “Ideal WON ups 
does not differ greatly for the two 8? sizes 
but while the high group jas adt 
qualities of skepticism and ШШ pd 
d su 
s the 
henr 
ick 


h 


the low emphasizes passivity an 
siveness. The high-scoring group "i 
"Average Woman" as much like t D 
selves. She has many aggressive, Pid 
ous, and stubborn qualities, gent 
nurturance and generosity аге sU 1100: 
slightly more than in the sell-conceP рег 
The low-scoring group, on the пап" 
hand, considers the “Average we 


, si "-———— 
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TABLE 8 


TABLE S—(continued) 


ADJECTIVES 
OF 


[TM M 


1 M9EFERENTIATING А GROUP 


EC: 


‘True False 
Blunt | | Affectionate, 
Competitive miling and 
Complaining yleasant 
Contiding Big hearted 
Cool and calculating Businesslike 
Critical of others Can't refuse to cooper- 
Dominating ate 

Charitable 


Easily fooled 

Forthright and critical 

Frequently disap- 
pointed 

Gloomy 

Hard boiled when 
necessary 

Hard to convince 

Has childlike trust 

Impatient 

Irritable 

Occasionally jealous 

Outspoken 

Resentful 

Resents being bossed 

Sarcastic 

Selfish 

Self-punishing 

Skeptical 

Snobbish 

Sometimes disrespect- 


fu 
Straightforward and 
direct 
Suspicious 
‘Too easily influenced 
‘Too lenient with others 
‘Touchy and eas ily hurt 


Wants everyone’s love 


Devoted follower 

Encouraging others 

Enthusiastic follower 

Firm but just 

Fond of everyone 

Forgives anything 

Gentle and reassuring 

Good leader 

Guiding 

Humble 

Likes everybody 

Makes a good impres- 
sion 

Managing 

Modest 

Obedient 

Plays it safe 

Respected 

Responsible and de- 
pendable 

Self-reliant and asser- 
tive 

Self-respecting 

Stern but fair 

‘Tender and soft- 
hearted 

Trusting 

Uncritical of others 

Unselfish 


Warm 


Ideal W 


oman 


‘True 


False 


Forgives anything | 

Forthright and critica 

Нага to convince 

Occasionally jealous 

Proud 

Skeptical 

Sometimes disrespect- 

, ful 

Tender and soft- 
hearted 


Apologetic 
Dusinesslike 
Humble 
Obedient 
Plays it sale 
Self-punishing 


Wants everyone's love 


fru 

Asking for trei 

e T 

Cool and calculating 

Critical of others 

ily fooled 

ily led 

Forgives anything 

Frequently dis- 
appointed 

Hard to convince 

Impatient 

Irritable 

Outspoken 

Oversympathetic 

Resentful 

Selfish 

Self-satisfied 

Shrewd and 

Skeptical 

Snobbish 

Soft touch 

Бореца disrespect- 
u 

Straightforward and 
direct 

Submissive 


Suspicious | 
Too easily influenced 


‘Touchy and easily hurt 


practical 


EV ve 

Alw ys smiling and 
pleasant 7 

Apologetic 

Big hearted 

Businesslike 

Encouraging others 

Fond of everyone 

Forceful ` 

Humble 

Independent 

Modest 

Obedient 

Responsible and de- 
pendable 

Self-reliant and asser- 
tive 

Unselfish 


Wants to be 1€ 


Ideal Man 

‘True False 
— Am 
Blunt Always smiling and 
Critical of others pleasant 
Dominating Apologetic 
Flattering Businesslike 
Forthright and critical | Can't refuse to cooper- 


Gives freely 

Hard to convince 

Occasionally jealous 

Sarcastic 

Skeptical 

Sometimes disrespect- 
ful 

Straightforward and 
direct 


ate 
Dependent 
Fond of everyone 
Humble 
Likes everybody 
Modest 
Obedient 
Plays it safe 
Resents being bossed 
Uncritical of others 
Wants everyone's love 


a The adjectives were 
themselves to describe 


"Ideal Man. 


used by the subjects 
their conceptions 0 
“Average Woman," an 


» N=64 in poth groups- 
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to possess fewer “negative” qualities. 
They see her as more affiliative and sub- 
missive than does the high-scoring group. 
Lastly, the "Ideal Man" presents con- 
siderable contrast. The low-scoring 
group stresses affiliative qualities, the 
high group aggressive and cynical quali- 
ties. 

Here again it is possible to see the 
rather "negative" self-picture of the high- 
J subjects-negative according to the 
common values of our culture—and the 
rather positive or pleasant self-picture 
of those low on J. And one may note 
considerable correspondence between 
the contents of these sell-pictures and the 
content of the J scale itself. "Blunt, ag- 
gressive" is like л Agaression, n Domi- 
nance, and л Defendance; "skeptical, 
distrustful" is like n Rejection, n Au- 
lonomy, n Defendance—and so on. 

The undoubted tendency to self-de- 
preciation on the part of the subjects 
high on J is rendered less ominous by 
the fact that the subjects themselves 
parently do not see their as 
tive” qualities as so bad; 


ap- 
ribed “nega- 

they think 
that ideally a woman ought to be skep- 
tical, autonomous, and perhaps some- 
what hard-boiled. 

Both groups seem largely 
by supplementary projection 
their conceptions of the 
Woman” and of the “Ideal Man.” That 
the high-J group see the ideal man as 
aggressive and cynical, like themselves, 
does not mean that they like men any 
less than do the low-J subjects; on the 
contrary, the J scale itself 
larly the n 


governed 
in giving 
Average 


‚ апа particu- 
Dominance items, showed 
that it was qualities like aggression and 
cynicism that high scorers find most 
attractive in men; such qualities seem 
to serve as stimuli for the exercise of 


the same tendencies in themselves, 
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D. Adjective Descriptions by 
Psychologists 

In all of the above attempts to parum 
the meaning of the J scale, scores on the 
new instrument have been related Ha 
other measures based on the subjects 
self-ratings. It would now seem desirable 
to consider scale scores in relation to an 
external criterion. 

The 50 alumna discussed in conn 
tion with Table 1 took part in a dem 
personality assessment project n 
which they were subjected to a wide 
variety of procedures, including — 
by staff members, Among the rating pre 
cedures used was a large adjective Eck 
list. Each of 5 raters independently 
checked in a list of Goo adjectives ese 
which applied to each of the s 
About 80 adjectives were found to whe 
criminate (using у? at the .o5 level) ыз 
14 highest scorers on the J scale fron 
the 14 lowest scorers, oh 

Adjectives most descriptive of the hig 
and of the low scorers are presented ix 
parallel columns, in the order of ран 
discriminating power, in Table 9- phen 
describe sharply contrasting patterns ү 
behavior. The most discriminating ud 
jectives—impulsive, irrepressible, re 
full of Pep, erratic, vigorous, icum и 
impatient, and the like—scem to an 
fairly vivid picture of impulse on 
in its objective aspects. And similarly re- 
the low end of the J distribution— 
tiring, silent, reliable, inhibited, 
served, stolid, and the like. We oe 
so far discussed the meaning of lis- 
Scores on the J scale, A study of the oh 
criminating adjectives suggests that pora 
scores may be obtained either H aed 
jects who are anxious and pat at 
or by subjects who are placid ane 
peace with themselves. tives 

It is interesting to note the adject 


ow 
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TABLE o 
ADJECTIVES CHECKED BY SSESSMENT RATERS 
To DESCRIBE 14 LOW AND 14 
Нісн SCORERS ON J 


High Scorers Low Scorers 
Big-boned Retiring 
Impulsive | Silent 
Irrepressible | Reliable 
Angular Inhibited 
Loud Reserved 
Гап Stolid 
Full of pep Humorless 
Erratic Shy 
Vigorous Withdrawn 
Restless Plain 
Impatient Faint-voiced 
Stylish Meek 
Uninhibited Painstaking 
Excitable Quiet 
Outgoing Weary 
Humorous Placid 
Pleasure-seeking Respected 
Argumentative Reticent |, 
Critical of others Uncommunicative 

Assertive | Thorough 
Egotistical Fearful | 
Charming Apathetic 
Long-legged | Mild 
Able to give orders | Passive 
Tactless Anxious 
Modest 

| Calm 
Frivolous Wan 
Immoderate Dignilied 
Worldly Patient 
Handsome Dutiful 
Witty Cautious 
Easy-going 
Sophisticated 
Bony 
Frank 
Self-centered 
Curious 


Aggressive 
Demanding 
Hasty 
High-strung 
Modish 

Good natured 
Tllogical 
Delightful 


descriptive of physical features that dis- 
tinguish the high scorers on J—big-boned, 
angular, tall, long-legged, bony. They 
ате enough to suggest the possibility ol 
а constitutional difference between the 
high and the low scorers. 


IV. SUMMARY AND INTERPRETATION 


The major findings that have been 


reported in this paper are the following. 

1. The scale developed for the meas- 
urement of impulse expression, the J 
scale, is statistically homogeneous, as 
judged from item-total correlations and 
reliability coefficients, but its constituent 
items are readily classifiable under the 
headings: Ascendance, Sensation, Endo- 
cathection and Intraception, Radical 
Sentiments. 

2. The scale clearly has to do both with 
impulse expression that is more or less 
spontaneous and free, and with impulse 
expression that is defensive or “driven.” 
These two tendencies appear to suffuse 
the scale as a whole rather than being 
separately represented by different items. 

3. Seniors score significantly higher 
on the scale than freshmen, and higher 
than alumnz in the age range 40-45. 
The difference between freshmen and 
alumnz is not significant. 

4. The scale correlates positively with 
Delinquency (De), Dominance Vegting 
in Women (M Do), a measure of femi- 
nini that accents the “inner че 
(MF 111), preference tor “activity” \ А 
figure-preference test, and a measure of 
developmental status based on empiri- 
cally determined differences between 
seniors and freshmen (VDS). lt corre- 
lates negatively with Social Responsi- 
bility (Re) and with a measure of femi- 
ninity that accents "passivity" (MF II). 
Correlations of the J scale with the E and 
Е scales of The Authoritarian Perso- 
nality did not differ significantly from 
zero. 

5. Scores on the J scale are positively 
associated with MMPI indices of expres- 
sion (rather than repression), degree of 
“psychopathology,” and tendency to use 
“psychotic” mechanisms of defense. 

6. Subjects scoring high and subjects 


scoring low on the ] scale differ in the 
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way they use an adjective check list to 
describe themselves, the "Ideal Woman," 
the “Average Woman,” and the “Ideal 
Man." The high scorers show more dif- 
ferentiation among these conceptions; 
they give a relatively “negative” descrip- 
tion of themselves and see themselves as 
more like their ideal man than like their 
ideal woman; aggressiveness and skepti- 
cism stand out in their conceptions of 
themselves and of the ideal man. 

7. When an adjective check list was 
used by 5 psychologists to describe the 
behavior of a group of 50 alumna, many 
more adjectives than would be expected 
by chance were found to discriminate 
subjects scoring high from subjects scor- 
ing low on the J scale. The adjectives 
distinguishing the two groups of subjects 
formed two self-consistent pictures, the 
one including “impulsive,” "irrepressi 
ble," "impatient," and the like, the other 
"reticent," "reserved," "responsible," and 
so on. 


How should these results be inter- 
preted? We are not prepared to offer 
a comprehensive theory. More research 
is needed, particularly intensive clinical 
studies of high and low scorers, and re- 
newed efforts to separate the sponta- 
neous from the defensive aspects of the 
tendency to impulse expression. We may 
indicate here, however, the directions 
of our thinking at the present time, 


Freud's (10) paper on "Libidinal Types" seems 
never to have attracted much attention, perhaps 
because it was too sketchy. He merely indicated 
the possibility of distinguishing Compulsive, 
Erotic, and Narcissistic types on the basis of the 
major direction of libidinal attachment—whether 
to internal agencies of control, to external ob- 
jects of love, or to the self—and he went on to 
stress the commonness of mixed types. Sanford 
(23) leaned heavily on this scheme in working 
out a typology of true criminals, He distin- 
guished the Anti-Social (Compulsive), who were 
capable of acting according to some principle, 
such as loyalty to a dissident group; the Pre- 
Social (Erotic), whose criminality proceeded from 
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their positive involvements (they were por. 
led") or from their negative involvements Ше › 
the objects of their love; and the Asocial, W of 
had no loyalties but were out “to take care O 
number опе” (Narci istic). . ame the 
It appea that according to this scheme nd 
"defensive" high scorers on the J scale wai 
belong to the Narcissistic type, or tliat үс al 
sism would be the main feature of their Шш 
strivings. The more “spontaneous” high scare 
would also have a strong measure of пата ism 
but this would be mixed with Erotic and К ПШ 
pulsive components, Some of the low 9 a 
probably the most extreme oncs, would T s 
the Compulsive type, while others would : 
primarily Erotic with Compulsive elements: 
We may consider first the defensive 
scorers, The hypothesis of central aid 
seems to be supported by what has been xo 
before concerning the J scale's accent on p 
seeking, self-indulgence If-defense, and E 
absence from it of items expressive of moral A 
cern or of positive feeling for other people. ` als; 
ford (23), in writing about the asocial emis 
distinguished between the manifestly “ego u 
cal,” who sought by exhibitionism and map 
lativeness to get something from other peor = 
and the “withdrawn narcissists,” the lone мае 
whose lives were dominated by fantas iens 
private schemes, ‘Tendencies of both the e fi 
would scem to be present in the subjects Те 
on J, though in less extreme degree than 1n ae 
criminals, One might suppose that there. VD 
disturbances in thc subjects’ carly relations ШШ 
objects, so that there was instability їп ridi 
introjections and distrust in their relations W or 
people, leading the subjects to strike ous Г 
themselves, Of course, striking out for БИ, 
by self-forwarding and scelf-indulging tendu: be- 
is not likely to raise self-esteem, not only the 
cause the subject must sooner or later bear ДО 
guilt for such behavior but because the panar not 
however successful in a practical way, ЙЕ is- 
influence the unconscious sources of the narcis? 
lic wound. igious 
Barron (3), in a clinical study of the relig 
belief. systems of the 50 alumn;e used in 
present research, came. to the conclusion в 
disappointment in the father and anger br 
him was the psychodynamic force which 1 im 
ап affirmation of atheism. ‘The atheistic “ai 
that there is no God is essentially a en aia 
benevolent supernal power, as well as а SE A 
tion of infantile wishes for love and for а jous- 
of strength external to oneself.” Since ле i 
ness is an important trend in the J SO pre 
may be fruitful to explore this line of ine 
tation here. What holds for irreligiousness for 
also hold for other Radical Sentiments апе 
other trends in the J scale. 


The hypothesis of disappointment in ОГ 


inst 
to 


ange 
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with the father is particularly relevant to our 
findings concerning masculinity-femininity in 
relation to J. A natural way for a girl to manage 
such feelings toward the father, or to respond to 
his loss through death or desertion, is by intro- 
jection of him. That such introjection was fai ly 
common among our high-] subjects would be 
one way of accounting for the substantial nega 
tive correlation. between this scale and MF П 
(passivity), and for the emphasis on activity, 
mastery, and daring in the J scale itself. But 
identification based on hostility and fear could 
not be expected to lead to the iblishment of 
stable institutions within the personality. High- 
J subjects, as suggested carlier, have relatively 
Unstable or perhaps unformed » identities; 
they are by no means content to be "like father" 
or like а man; her, it seems, they want to 
carry on certain kinds of relations with men. 
The correlation between J and the Maslow 
items, the ascription by high-] ub- 


Dominance 


jects of aggression and cynicism to the “Ideal 
Man," and the contents in the J scale itself sug- 


gest that although she wants to punish and 
wants them to 


dominate men, she neverthele: 
be tough, like herself, so that he 
not lead to any catastrophic consequences 
Premature sex experiences might be 
in producing some of the attitudes just described. 
‘Yo hypothesize such experience would be also 
to give due attention to the element of Sensa- 
tion in the J scale, and to the element of De- 
linquency, Adolescent gratification at the genital 
level creates a pattern of gratifi king 
that is not to be given up ily, even though 
its maintenance requires strenuous efforts to 
keep conscience in abeyance. 
nument in, and anger with, the father 


factor 


hypothesizing as a major source of high-J scores 
experiences leading to disturbances in introjec- 
tion and in relations with primary objects, such 
that the subject was forced to look to her self— 
to its advancement and to its protection. Such 
experiences usually occur first in a subjects re- 
lations with the mother. Indeed it seems likely 
that girls who are led into such relations with 
the father as we have described have first to deal 
with a mother who is relatively cold and capri- 
cious, or weak and inconsequential, or inflexibly 
strict. 

It is to very early relations with the mother 
that one ordinarily looks for the circumstances 
leading to the differentiation in Compulsive, 
Erotic, and Narcissistic types of libidinal attach- 
ments and to the beginnings of those mechanisms 
of defense which scem to accompany them, €g. 
reaction. formation in the Compulsive, repres- 
Sion in the Erotic, denial and distortion of reality 
in the Narcissistic. Our high-J subjects may 


have learned very early to prefer either action 
against the real world or denial and distortion 
of it as mechanisms of defense. It seems unlikely 
that the traumata of their early childhoods were 
severe enough to generate strong psychotic po- 
tential, but they may have been important 
enough to determine the type of response 10 
later crises. It might be that in many of these 
subjects a nar ic wound occurred at a time 

when a fairly substantial ego structure had been 

formed, so that they could feel, consciously, let 

down, disillusioned, embittered, insecure, and 

could determine to retaliate, to get what was 

rightfully their own, to make some radical 

changes in the world, to see to it that they were 

not again caught off guard. 

It cannot be argued that all subjects who suffer 
disturbances in their carly object relations be- 
come fixated upon the sort of defenses that have 
just been described. Such disturbances are also 
commonly regarded as major sources of depres- 
sion, and very little is known of what determines 
the choice of reaction. At any rate, we are hy- 
pothesizing that just as action and denial be- 
come the defensive high-J scorer's preferred. re- 
action to disturbances in early object relations, 
so depression becomes preferred in similar cir- 
cumstances by the Compulsives who are extreme 
low-] scorers. 

Concerning the more spontancous high-J 
scorers it might be suggested that the mother 
herself was of the high-] type and had a free, 
give-and-take relationship with the father, She 
was not so cold or capricious as to create dis- 
turbances in the subjects early relations with 
her; and identification with her occurred later. 
The subject strove for the same type of relation- 
ship with the father as the mother had, and this 
was not sternly opposed from either side, The 
father was free enough to welcome responsive- 
ness from his daughter, but able to require that 
her impulsive aims be sublimated. In these 
circumstances a girl could acquire a durable, if 
mild, superego and a fair measure of social 
feeling; and enough spirit to resist by action, or 
by provocativeness, the frustrations of a pro- 
longed adolescence. 

How are we to interpret the fact that scores 
on the J scale seem to increase during the col- 
lege y ? Two hypotheses may be suggested, 
the one having to do with increases in the more 
spontaneous aspects of J, the other with circum- 
stances making for increases in the defensive 
types of impulse-expression, 

It may be supposed that the college experience 
tends to bring about some lifting of repression; 
and that it does this by offering means for learn- 
ing control at the same time that it stirs up the 
impulse life. If we have made a case for self- 
seeking and defensiveness as major features of 
the scale, we may also make a case for its ex- 
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pression of “self-actualization” in Maslow's (21) 
sense of this term. Many items of the scale scem 
to express autonomy, realism, acceptance of self 
and others, enjoyment of sensory pleasures, and 
spontaneity. á 
Fromm (11) and Maslow (21) have written 
about “selfishness” and genuine self-love as op- 
posites, in the sense that the former springs 
from a lack of the latter. We have followed this 
line of thought in some of our formulations 
above. Now, however, we would propose that 
self-actualization may as well grow out of a 
personality organization of which selfishness, or 
the kind of defensiveness that we have described, 
is an integral part. “Genuine self-love” would 
seem to involve no less “self” and no less libidinal 
attachment than selfishness; the difference be- 
tween them is that the object of attachment is, 
in the former, more worthy of it, and that there is 
enough sureness of that object and hence enough 
freedom so that, as in the case of Captain Shot- 
over, the self-love may spill over onto the out- 
side world. In other words, we would still a үң 
that high scorers on the J scale were prim 
Narcissistic, rather than Compulsive or atic, 
even though they owed their highness to a prefer- 
ence for the more sensible items and showed 
other signs of maturity, Growth toward self- 
actualization may occur as well in people who 
started from a Narcissistic orientation as in those 
who started from other orientations, It may be 
suggested, indeed, that subjects of the Compul- 
sive or Erotic types have to pass through a nar- 
cissistic phase on the road to maturity, 
hat many seniors are in a narcissistic phase, 
and that this is largely provoked by the situa 
tion in which they find themselves, is our second 


hypothesis to account for their relatively high 
J scores. 


We certainly do not wish to say that the sen- 
iors as а group are more self-actualized, or freer 
in the best sense, than the alumni; rather, we 
should say that they are striving for self-actuali- 
zation, Many seniors are in a situation of having 
thrown off traditional values without having 
fully established others of their own, of having 
loosened long.standing inner controls at a. time 
when new experiences have to be integrated, of 
having rejected old identities at the very time 
when important decisions have to be made, We 
should not be surprised, then, if their strivings 
for self-actualization have an aspect of insistence, 
that they tend to be rebellious rather than au- 
tonomous, dominating rather than self-assured, 
cynical rather than realistic, hungry for sensa- 
tions rather than able to enjoy them in a re- 
laxed way. Among the hypothesized determi- 
nants of highness of the J scale, then, is a par- 
ticular kind of developmental crisis, to which 
some subjects are more susceptible than others, 


and through which different individuals pass at 
different times. r 

Most of the hypotheses suggested here call a 
comprehensive studies of life tory bene 
in relation to J, and for intensive clinical stuc ies 
of individuals scoring at the extremes on the 
scale. At the same time, and perhaps more m 
mediately, the need is for studies aimed Т 
differentiation among high and among low 
scorers on J. We have already expressed our К 
terest in the possibility that the former DE 
be divided into the “free” and the defensive, the 
latter into the constricted and the pel 
perhaps, the Compulsive and the Erotic. EN 
terion groups of these kinds will, of course, e 
10 be set up on the basis of clinical studies а! 
used in connection with item analysis of the 1 
scale. We shall also be able to use our sample 
of 50 alumnie for this purpose, [ the 

Another possibility is to take advantage © ai 
absence of correlation between J and the F Sed 
This means that we undoubtedly have in m 
college samples large numbers of subjects P üt 
are high on J and high on F, high on J PY 
low on F, and so on. Since much is known abou” 
the F scale, studies of this kind should nd 
some light on the meaning of J. But we may ie 
have here a means for making significant dil M 
entiations among extreme scorcrs on För а 
ample, the woman who is high on J and Ё ate 
well be one of those phallic, sadomasochis 
kinds of women, whom we see occasionally оп ity 
political scene, who project. their masculine 
onto leaders or generals to whom they eti 
submit masochistically, High or moderately PM 
Scores on J but low scores on F, on the oF 0 
hand, might turn out to be a good indication 
genuine freedom. 

Our most immediate concern is with dev 
ment during the college years. We have d 
men who are high and freshmen who bM йб, 
on J. How do the two groups change, if they 
under the impact of education, and what à 
circumstances of such changes? To tak 
other way around, did the seniors who av nen 
standingly high on J start out as fresh 
already fairly high, or has there been а 
“breaking loose” from a pattern of an opr A 
kind? Have seniors who are low or middle Sar 
already attained a good measure of emotio! a 
control, or does there remain a potential one 
“lifting of repression” at some later time? 


Б : єз оп. 
Suggestion, made above, that high scores от“ 
may reflect a a n 


clop* 
resh- 


posite 


developmental crisis that 


1 ay have 
mon in college seniors, means that we may an 
à particularly good opportunity for investiga! ie 
in a 


“momentary situation" as it were 

dynamics of impulse expression. like 
Finally, the J scale as we sce it measure, j, 

the F scale, д pattern of attitudes that iS ? 


е 


mately 


IMPULSE EXPRESSION AS A VARIABLE OF PERSONALITY 21 


tied to unconscious sources in the per- 


sonality. For this reason we should expect it to 
correlate significantly with many other measures 
of performance and of personality functions. We 
should not, however, expect these correlations to 
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be very high, just because J, standing at the level 


of 
in 


attitude, may be expressed in overt behavior 
different ways and have different sources 


within the personality. 
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Fergus Falls State Hospital, Minnesota 


A of the literature reveals no 
4 systematic inquiry into the behav- 
ioral characteristics of an entire hospital 
population. Yet this behavior is of con- 
cern both to those attempting treatment 
and to society as a whole. The actions of 
commitment courts, county welfare 
boards, relatives, clinicians, and scientists 
are based at least in part on opinions 
about the behavior of these patients. 
These opinions may or may not be ac- 
curate. There is urgent need for objec- 
tive study of the actual behavior of men- 


tal hospital populations, so that various 
therapeuti rograms can be better 
geared to variation in patient abilities 
and needs. 

In the past, scientists in this field have gen- 
erally limited their investigations to a small 
segment of those exhibiting behavior pathology. 
The recently admitted patient has been the 
subject of most state hospital research. Various 
generalizations have been made, without ade- 
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quate research, about the behavior of continued- 
treatment patients. 

This is a report of an objective longitudinal 
study of the behavioral characteristics of an 
entire state hospital population. The authors 
hope that the results of this study may assist 
hospital personnel in planning therapeutic re- 
gimes, and ultimately aid the public in better 
understanding the behavior of the mentally ill. 
A further expectation is that the study will be 
of value in supporting or weakening various 
hypotheses. 

The large amount of data, the number of 
factors, and the complex interrelations make it 
imperative that this paper be a “bird's-eye view.” 
It is hoped that further and more detailed anal- 
yses of the data will be made. 


Tue НоѕРІТАІ, 


State Hospital is a 2,000-bed 
1 located on the outskirts of a 
city of 12,000. Of the 1,086 acres that comprise 
the hospital grounds, 725 аге farmed. Almost 
none of the 455 authorized positions are vacant, 
An average of 60 separations occur per year. 
Approximately 309 of the employees are pri- 
marily concerned with patient therapy and ward 
administration. The others are employed pri- 
marily for other purposes, C£. kitchen, supply, 
maintenance, engineering, ctc. Many of these 
people also have a great deal of patient contact 
(for example, about 75 patients are working in 
the main kitchens) . 

The budget for the fiscal year 1 July 1954 to 
1 July 1955 was $2,044,824. No major construc- 
tion costs are included in this representative 
figure. 

The hospital has three integrated (team ap- 
proach) programs treating the following types of 
patients: recent admissions; regressed, long- 

1 recently lobotomized patients; and 


term, and 
continent, ambulatory seniles. Other wards in 


the hopsital are screened periodically to provide 
patients for the latter two programs. 


Fergus Falls 
mental hospita 


SUBJECTS 
The subjects were the entire resident 


Bureau Edni. "sy. Research 
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population of the Fergus Falls State Hos- 
pital. Two surveys were made: one in 
the first week of April 1951 (N = 1,932): 
and another, two years later, in the third 
week of May 1953 (N = 1,925). Of the 
patients in the first study, 1,481 were in- 
cluded in the second. Patients absent 
from the hospital for the whole week for 
any reason were not included. 

The patients are drawn from a pre- 
dominantly rural area of Minnesota. 
The largest city has a population of 
28,410. (There are five cities with a pop- 
ulation of over 10,000 and 26 of over 
2,500.) The principal occupation of this 
portion of the state is agriculture. 


PROCEDURE 


Data were gathered on a total of twelve be- 
havioral and nine case-history items. The nine 
case-history items are: Diagnosis, Present Age, 
Sex, Marital Status, Religious Affiliation, Physi- 
cally Debilitating Condition, Total Length of 
Hospitalization, Education, and Age at First Ad- 
mission, 


Eleven of the behavioral items were derived 
from ratings of the areas on the L-M Fergus Falls 
Behavior Rating Scale, hereafter referred to as 
the L-M Scale (2, 3). The twelfth is an average 
of the ratings on these areas. The eleven areas 
are: A, Work; В, Eating Behavior; С, Behavior 
Toward Other Patients; D, Behavior Toward 
Psychiatric Aides and Nurses; E, Behavior 
Toward Doctors, Social Workers, and Psycholo- 
gists; F, Attitude to Electric or Insulin Therapy; 
G, Participation in Occupational Therapy and 
Recreational Therapy; H, Attention to Dress and 
Person; I, Psychomotor Activity; J, Speech; 
K, Toilet Behavior? 

Both in 1951 and in 1953, each patient was 
rated independently by two people on the ward 
(psychiatric aides and/or nurses). If the raters 
disagreed by three or more points with respect 
to their ratings of a patient in a given area, the 
rating on this area was considered invalid. The 
average of the two ratings for the area Was used 
as the best estimate of the patient’s behavior, In 
addition, the mean for all the areas (a more 
reliable measure) was computed for each patient. 
The patient's behavior was rated for the week 
prior to the date of the rating. Two weeks fol- 
lowing the initial notice, all ratings were com- 
pleted. 


and 


* The scale is reproduced in its entirety in the 
Appendix. 


This scale does not give information with 
regard to thought processes, except insofar as 
inferences can be made. " 

There are some differences between the nta 
collected in 195: and in 1953. arising pee 
(a) minor differences in definition: and (b) i m 
thorough examination of the patient's КОҢ S 
1953. No changes in definition or MERED ar 
curred with respect to Present Age, Age üt ЭЙ 
Admission, Diagnosis, Sex, Marital tus, ah 
Length of Hospitalization. Age at First аш 
sion and Length of Hospitalization refer. T 
Minnesota State Mental Hospitals, Diagnosis 15 
the latest one made by the hospital. "T 

Religion was determined more accurately sid 
1953. The differences in definition of eee 
level and physically debilitating factors wi 
discussed later, hos- 

Some comparisons are made between ШЕ ate 
pital population and the population of the xm 
of Minnesota as determined by the United Sta 8 
Census Bureau for the year 19; (7). pne ee 
counties in the 1950 admission district of Е 
Fergus Falls State Hosp were used to CC 
termine the characteristics of the normal ч 
lation. The population of these counties wa 
503 of whom 263,787 were males and 239.53 
were females. "These figures include only s 
Broups over 15, since hospital admissions on 
rarely below this age. The hospital рар 
is included in the 1950 census figures used j 
this comparison, Therefore differences a 
between the census and hospital populations W! 
have maximal interpretive significance. di- 

Since these populations are biased in the A 
rection of rural areas, caution must be used A 
applying the results of this study to pred 
nantly urban hospital populations. The chara 


teristics of a rural, 
Sc 


fidwestern people, many © 
andinavian descent and Lutheran faith, тау 
differ quantitatively from those of other көш, 
In some instances, а direct comparison Bee 
the hospital population and the normal pon ave 
tion gives ambiguous results, The authors p 
reduced the ambiguity of these results by } 
use of certain corrections, 


REsuLTS 

The number of patients rated on ue 
first census Was 1,932, of whom 998 sep 
male and 934 were female. Between 8 
time of the first census in April of 195" 
апа the second census? in May of € 
а total of 451 patients left the hosp 
This figure does not include patien 


" shall 


* Hereinafter, the 1951 behavioral censu er 


49 cens 
be referred to as Census I, and the 1953 © 
as Census II, 


b 
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TABLE т 
Present AGE—HosPrTAL POPULATION 

| Census I Total 

| | D Adm. 
Age Male Female Census 1 Census П ) betw. 

(№008) (N=034) | (Y= 1032) (N- 1923 (y ® cens. 
N-r4B) (N =444) 

| % % % % % % 
16-25 2.7 2.6 2.6 2.8 1.1 8.3 8.5 
26-35 11.4 11.7 11.6 8.7 7.8 II.7 12.8 
36-45 20.7 18.0 19.4 19.7 21.3 14.2 20.9 
46-55 21.0 18.0 20.5 20.4 22.5 13:5 10.9 
56-65 19.1 23.2 20.0 19.1 21.0 12.6 22.8 
66-75 14.4 14.1 14-3 16.0 16.1 16.0 0.0 
76-up | 11.7 11.2 11.4 13.2 10.1 23.7 5.2 
Undet. Е 3 .2 E E 
Mean 53.63 54.10 53-85 55.27 54-96 65.33 49.72 
Median 52.18 54.78 53-29 54-68 54-57 57.50 50.11 


Note.—Means and medians were compu 
this table. The more detailed tables are 
are taken from Census Il. 
who entered the hospital subsequent to 
Census I and departed prior to Census 
II; it does include 182 deaths and 58 
transfers that occurred during this time. 
At the time of Census II, a total of 211 
patients rated on Census I were absent 
from the hospital on either provisional 
or final discharge. The number of pa- 
tients rated on Census II was 1,925; of 
whom ggi were male and 984 were fe- 
male. 

Hereafter, the follo 
used to describe patient groups: 
sions"—the 444 patients rated only on 
Census II; "discharges"—the 211 patients 
rated only on Census i “permanent pop- 
ulation"—the 1,481 patients rated on 
both behavioral censuses. 

A discussion of the population of Fer- 
gus Falls State Hospital will be based 
primarily on the data from Census 1, 


except where a more accurate determi- 
) Census II, or when 
n the two 


wing terms will be 
"admis- 


nation was made it 
comparisons are made betwee! 


censuses, 


Present Age 


with 
There was no special problem connected Р 


ted from data grouped into finer categories than those in 
available upon request. Data for the permanent population 


the determination of age. For the past 25 years 
or so it has been customary to collect the birth 
date. Occasionally, there is some question, espe- 
cially with older patients, or p: ients who were 
admitted a number of years ago, as to their exact 
age. The age of the patients is determined to the 
nearest year (for example, a patient who is 6o 
years, six months, and one day is called 61). 

Approximately 36 per cent of the hospital 
population on Census I is 61 years of age or 
older, as compared with 39 per cent on Census 
Il. The mean age of the patients on the second 
census is greater by 1.42 years. This difference is 
reliable‘ and indicates a trend toward aging of 
the hospital population, Discharged patients are, 
on the average, younger than the hospital popu- 
lation. 

The fact that the age shift is significant in 
only a two-year period is of extreme interest in 
terms of the direction of therapy. If this trend 
continues, greater attention must be paid to the 
care of a population which differs markedly from 
the younger psychotic patient, The older person 
has interests different from those of the younger 
person, and must have different types of treat- 
ments and activities in which to participate. 
This also means that the hospital has a smaller 
number of patients who are capable of perform- 
ing arduous physical tasks, Therefore an in- 
creased number of employed personnel must be 
available to carry on the routine work of the 
institution. In addition, the physical layout of 


س 


‘In this paper differences reported as reliable 
uniformly meet the 1%, level of confidence. 
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TABLE 2 
AGE AT First ADMISSION M 
Census I Total EM 
I Adm. Disch. 
2 . Я etw 
Age Male Female Census I — Census П Dog реи, нь 
(N=998) (№=034) | (№=1,032) (N=1,925) (N=1,481) (aaa) een 
к. 98 % % % % % g 
16-25° 20.7 18.8 19.8 20.0 22.0 13-7 ү 
26-35 25.2 28.0 26.5 25.0 28.4 13.¢ 15.0 
36-45 17.7 19.4 18.4 18.5 19.8 14-2 э-ү 
40-55 13-5 14.8 14.1 14.2 14.6 12.9 36.5 
56-65 0.7 8.2 9.0 7.8 7.2 9 6:6 
66-75 6.8 5.0 6.0 7.I 4.5 15.5 ja 
76-up 6.2 $53 5.8 7:2 3.4 20.1 
Undet. .2 5 “4 .2 T .2 
% ‚72 
Меап 41.75 40.52 41.15 41.94 38.71 52.14 erem 
Median | 37.45 37.00 37.23 37.70 35.36 52.50 Е 


^ Patients admitted before the age of 16 constitute approximately 1% of this age group. 


the wards must be different in order to accom- 
modate patients whose needs and capabilities 
differ from those of a younger group. 


Age at First Admission 


There were no special problems in the de- 
termination of age at first admission, In ver 
instances in older records the age was not 
tioned for a short hospitalization in a 


TABLE 3 


AGE ОЕ ADMISSIONS TO FERGUS FALLS 
State HOSPITAL 


y таге 
men- 
Minnesota 


1 June 
1925 т June 1951 
through through" 
31 May 31 May 1952 
1926 
Age Readmissions 
First First (N= 129) 
adm. adm. Е эзне 
eer = ge at Age at 
(N= 428) (N 375) ISt present 
adm. adm. 
% % % % 
16-25 16.6 9.6 27.1 7.0 
26-35 22.4 13.1 15.5 10.8 
36-45 22.0 13.9 20.2 26.4 
46-55 16.4 15.7 21.7 22.5 
56—65 10.0 1353 8.5 17.8 
66-75 8.2 16.8 Balk 8.5 
76-up 4.2 17.I 3.1 6.2 
Undet. E 25 .8 .8 
Mean 42.43 53-01 | 30.37 48.01 
Median 40.99 54.17 | 38.14 48.18 


State Mental Hospital. The data in Table ind 
dicate that the greatest percentage of per ы 
(26.5%) occurs between the ages of 26 an tion 
inclusive. The Census II hospital popul ү» 
shows а statistically reliable shift toward an © n» 
age at first admission (mean, Census mh be- 
mean, Census II = 41.9). Patients admitte but 
tween the censuses (і.е, rated on Census ЇЇ, et 
not on Census I), average 54.1 years of RS on 
first admission. The "permanent" population, c 
the other hand, has a mean first admission A8 
of 38.7 years. issi 
The tomparatväy high age at first айша. 
of those admitted between censuses calls per- 
comment. These are recent first admissions. fact 
haps admissions to state hospitals are 1n ed 
older, reflecting the increased number of А 
in the normal population, On the other ha er 
because of longer life expectancy, youre 
"chronic" admissions remain in the soe 
longer than older "chronic" admissions em 
hence аге more apt to be counted in à cen 
Perhaps younger admissions have ponet ath) 
noses (for any type of discharge, including йе = 
than do older admissions, These items аге eg 
cussed later in this paper. Subtle changes nges 
Society's attitudes toward the aged, and pe in 
in modern housing, may also be Supr 
the increased admission age at this hospita- 


B . 1 Da 
Consecutive Admissions During О 
Year 

- e- 
Admission ages for a one-year admission n 
riod are given in Table 3. No special pro wing 
were encountered in this determination. T aci" 
this one-year period the average age ° 


——^ 
t 


CENSUS OF 


15- 20- 25- 
19 2, 29 2 


e of normal populat 


@— — —@ Present age of pat 
^ Аве at first admission of pat 
dmissions from 


Fic. 1. Present age of normal рор 
first admission 0 


readmission, was 49, and 
This 


admission Was 54. Th 
the first admis- 
п Census П 


шыш, at the time of 
Байы ame age of first i 
йола артеез substantially with 
\ (але ge of the patien 
feat ission between cen: 
Ma admissions from 1 JU 
th у 1926 (42.4 years) agrees V 
е age of admission of P tudied on 


suses). 
ne 1925 


A HOSPITAL POPULATION 


30- 35- 4O- 45- 


ulation compared with prese 
f hospital patients. 


Gt 


75- 85- 
84 over 


70- 


55- 60- 65- 
Th 


59 6 69 


jon 15 years ог older. N=503,325. 


pital on Census I, N=1928- 
he hospital on Census I. N=1920. 


1 June 1951 through 31 May 1952. N-373 


nt age and age at 


Census I (41.2 years). 
Other data support the admission-age trends 


noted at the hospital. For example, th 

age of first admissions to all Minnes pies 
mental hospitals for the fiscal year Pe ent 
53.8 years. In this hospital, ees к Ea 
for the years 1925 and 1951 show that thi je 
centage of admissions over age 65 пына. ов 
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12.4% to 33.9% while the percentage over age 
45 increased from 38.8% to 62.9%. Also, U. 5. 
Census figures for 1920 and 1950 (7) indicate | 
that the percentage of people over 65 (in Min- 
nesota) increased from 4.7% to 9.0% while the 
percentage over 45 increased from 20.8% to 


304%. 


° 


=$16,о76) 


Total 
1 
28.4 
3 
7 


Female 


Age Comparisons | 


Normal Population® 


Figure 1 illustrates age curves for various 
populations. The Census I hospital population 
has a fairly constant percentage of patients be- 
tween ages 35 and 7o. On the other hand, the 
normal population has a steadily declining per- | 
centage of people in each age group from 20 
through 85. 

In the hospital population of Census I the 
majority of first admissions occur between the 
ages of 20 and 4o. First admission ages for the 
hospital population cannot be compared directly 
with age groups in the normal population be- 
cause the mean age of the normal population, at 
the time of admission of many of our patients 
(in the ‘gos, '30s, and '40s), was younger than it 
is now, 

An estimate of the rate of admission from the NES m Га 
age groups in the normal population (ї.е., the 38 
rural counties in the hospital receiving district) 
is best obtained by a comparison with the ages 
of consecutive first admissions to the hospital. d 
During the one-year period 1 June 1951 through m 
31 May 1952, 504 patients werc admitted to this = 
hospital. Of these patients, 25.4% had previous 
admissions and were excluded. The 375 consecu- 
tive first admissions are, on the average, much 
older on admission than the patients rated on 
the hospital censuses. Considering the age dis- 
tribution in the normal population, it appears 
that first admission to the mental hospital in 
the present study is, currently, most apt to occur 
when an individual is over 50 years of age. | 

The hospital has a lower percentage of people 
below the age of 35 than does the population-at- 
large (12% vs. 39.5%). From ages 45 to 85-and- 
over the hospital population has a relatively 
higher percentage than does the normal popu- 
lation (68.1% апа 42.3%, respectively). Since the 
most frequent ages at first admission for the 
census population are 20 to 44, and since the pa- | pu 
tients have been in the hospital for some timc, 
one would expect that they would distribute 
themselves in the present older age range, 

Age-corrected samples are used in a compari- 
son of items where age differentials must be con- 
sidered. These age-corrections will be discussed 
later. 


age) 


Male 
(N = 270,280) (№ = 245,790) (N 
A % 
33.8 22.3 
от 67.0 
І 10.7 
ت‎ 


| 
| 


(corrected for А 
Compared with Normal Population 


Yo 


Total 
(У =201) 


Census II Patient Sample 


% 


Hospital Sample 
Female 
(N =130) 
cr 
+3 
3 
4 
10 
.6 


152) 


% 


Male 


211) | CN 
| 
| 
К. — 


сепз. 
(N= 
% 
37 
5 
1 
5 
o 


= 3.44) 


r5 
e 


cens. 


MARITAL STATUS 
(N 


1,481) 


% 


Total 
Perm. 
рор, 
59. 

2 


Hospital Population 


Census II 


1.9032)(N =1.025)(y 


î ls ae 


% 


Census Т 


| 


% 


Female 
(N =034) 
c 


£ 
a 
о 


Male 
(N =008) 


| 
| 
| 
| 
| 


Marital Status 


Status 
Single 
Married 
Separated 
Divorced 
Widowed 
Undet. 


There were no important differences between 
Census I and Census II regarding the marital 


® Includes all age groups 14 years or over. 
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TABLE 5 
RELIGIOUS PREFERENCE 


Hospital Population 
Census 11 | Total 
Religion Adm. Disch. 
Male Female. | CensusI Census II Teens betw: betw. 
(N =091) (N=034) |(V= 1932) (N =1,925) (Nar \8т) qn ) 
* s N —444 
% % % % % % 
Protestant 64.4 69.5 61.8 66.9 65.0 73.0 38.8 
Catholic 24.4 24.7 `6 24.0 25.2 22.5 ө 
Jewish it л E E »x 
Other 8.7 4.6 1.2 6.7 7.9 2.7 зл 
None +2 8.7 I .2 32.1 
Undet. 2.2 I.I 3.6 1.6 1.6 1.8 


The differences between the 


status of patients. е t 
hospital population 


normals and the Census I 1 
are all statistically reliable. A 
The hospita population has a much higher 
percentage of older people than the normal pop- 
ulation. To render comparisons more meaning- 
ful, a correction. was made for the age factor. 
These and all subsequent age-corrections were 
made by randomly selecting ап appropriate num- 
ber of patients from each age-group among the 
patients to approximate the age distribution of 
the normal population. The number of patients 
in the age-group 15 to 19 Was too small to permit 
the drawing of an adequate sample. The actual 
frequency in this group is 13, while the theoreti- 
cal frequency for а sample in this айе ор 
would be 36, Except for this age-group, UTE 
sample is adequate. { 
On Census t the majority (55 ^) of Mas pa- 
tients are single, compared to only 28% of п 
normals, The age-corrected patient sample nas 
а much higher proportion of single persons 
(69.807) than either the normal ог the B 
rected hospital populations. It may be that ШЛУ 
patients do not marry because of seclusive P of 
sonalities, passivity, the adverse reactions um 
others to them, or other unattractive tee 
factors, Some patients are removed from the 


environment before they reach marriageable BER. 
On Census I, almost 157% of the hospital end 
lation is widowed or divorced, comparec wi В 

population. In the age 


only 8%, of the normal 

10 ivi te- 
corrected sample, the widowed-and-divorced en 
Богу is equal to that of the normal РОР state 
Detailed 1950 census data for the €" si ee 
(7) indicate that 4.1% 6% 
1.695, are divorced, while 100 /o 

6% а , E er- 
Widowed and 1.9% ате divorced. E еа. 
centages apply to the ро =ч 18 arent that 
ing district of the hospital, it !5 арр her per- 
the hospital population has а much P liminary 
centage of divorced people- (Ehe E categories 
U. S. Census figures for 195° grove at figures are 
widowed and divorced. Detailed final fig 
not available for the receiving 


hospital.) 


pulation 


Religious Preference 


Religious preference was determined from 
religion as given on the commitment paper or 
from the patients own statement at the time 
of admission. There are instances in which the 
religious preference of the patient is not stated 
on the commitment papers, and there are also 
cases in which the patient is unable or unwilling 
at the time of admission to give his religious 
preference. Census II figures are quoted in Table 
5 since this census has a more thorough check of 
the patient's record if religion was not listed on 
the admission note. 

The percentages of church members of the 
predominant sects in Minnesota were taken from 
the 1936 Religious Census (6). These figures 
were determined from the membership rolls of 
the church, rather than by the questioning of 
individuals. Compared with the hospital popula- 
tion, more people in the normal population are 
classified as having no religion. The Catholic 
church membership as given agreed very closely 
with the percentage of Catholics found in the 
hospital. The percentage of Protestant church 
members outside the hospital was lower than was 
the percentage in the hospital. In the community 
a Protestant church member will be dropped 
from the church rolls if his attendance is irregu- 
lar, since the church must pay a certain sum 
per member into the national organization. 
Many people may not be registered as members, 
but, if asked, would state that they were. In the 
Roman Catholic church, a person who has been 
a Catholic is regarded as such until his defection 
becomes “public knowledge,” whether or not 
the member is active. 

Because of these differences in determination 
of religion the percentage figures for normal 
population and hospital population must not be 
compared directly, Catholic percentages agree 
closely in the normal and hospital populations. 
It is likely that Protestant percentages are more 
in agreement than the unexplained data indi- 


cate. 
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TABLE 6 
PHYSICAL DEBILITATION 
Hospital Population Census П Patient Sample 
(corrected for age) 
Census I | Total Compared with Normal 
Ad Disch. Population 
Status Peri m. h. 
Male Female Census I Census II betw. betw. = 1 
= = = = pop. h cens. Hospital Normal 
а д LC I M AN m 
% % % % % % % % % 
Without 85.0 84.6 84.8 85.6 88.6 75.4 92.4 89-4 ® 
With 15.0 15.4 15.2 14.4 11-4 24.0 7 то. 


Physically Debilitating Conditions 


Defining physical disability was difficult, In 
Census I all patients whose behavior was im- 
paired because of physical factors were counted 
as debilitated. Chronicity of the illness was not 
considered. In Census П, people with tempo- 
rary disability were not included. For this reason 
patients recovering from operations, broken 
limbs, or bedridden with acute short-term ill- 
nesses, were not counted as being physically 
handicapped. If the illness was severe enough to 
be noted on the physical examination, or if it 
was felt by a physician to be chronic rather than 
acute, or if the illness would permanently limit 
the behavior of the patient, it was classed asa 
physically debilitating factor. Amputation, 
blindness in one eye, or partial uncorrectible 
difficulty in seeing or hearing (provided these 
were serious enough to be mentioned in the 
notes on physical factors) were all counted as 
physical handicaps. Epileptics were not counted 
as handicapped unless their condition was so 
severe that it necessitated their being bedridden. 
Patients who suffered from a partial paralysis, 
or from a marked loss of a sense of balance were 
classed as physically handicapped. The figures 
for the normal population were derived from the 
number of beds in medical hospitals and in 
rest homes. This figure does not include all of 
the chronically disabled people in the com- 
munity, many of whom remain in private homes, 
However, the disparity between those listed as 
disabled in the population at large and those 
in the hospital population seems too great to 
be accounted for wholly on this basis. In the 
age-corrected sample (previously used in Table 
4) 10.7% of the patients are classified as debili- 
tated. The disparity between the normal and 
hospital populations remains great. 

The aged frequently have physical debilita- 
tion and personality change occurring in combi- 
nation, and the high percentage of aged in the 
hospital population partially accounts for the 
high frequency of physical disability. In the 
younger patient the presence of a physical 
handicap may result in certain unfavorable 
changes in personality which necessitates institu- 


tionalization in a mental hospital. People who 
have a physical handicap and an unstable por 
sonality tend to remain in the hospital rather 
than attempt a marginal adjustment on the 
outside. Perhaps all of these factors contribute 
to the relatively large number of physically 
handicapped found in the hospital. 


Length of Hospitalization 


The data for Table 7 were taken from Census 
I. It should be noted that the bulk of the 
population is made up of so-called chronic, long- 
term patients, and that, despite the relatively 
large number of admissions which this hospital 
receives in the course of a year, only 14% of the 
population has been here less than two years. 
This figure includes approximately 100 recent 
admissions, 

The mean length of hospitalization of the 
census population is 11.22 years, The patients 
who stay two years or more (86%, of our рори" 
lation) present the greatest therapeutic challenge- 
Even though present treatment methods are 
inadequate for them they accounted for 35% 
of discharges between censuses, The number О 
patients hospitalized between six and twenty 
years tends to remain relatively constant. After 
this time a sharp decline is found, which proba- 
bly reflects the increasing death rate of those 
patients. Also of interest is the difference an 
mean length of hospitalization between males 
and females. This difference, which is reliable, 
may reflect the slightly longer life span of the 
female, since females are approximately the same 
age as males on first admission. Perhaps, also, 
females are sicker or slower to recover than 
males. 


Educational Level 


Educational level in Table 8 is based primarily 
on Census II since definitions of the levels agree 
тоге closely with those of the О. S. Census 
Bureau, АП "common school" educations аге 
Placed in the 5-to-7-year group. (In Census 
"common school" educations were regarded as 
equivalent to graduation from the eighth grade 
Since only six years of Schooling were available 
in many areas of this state or in foreign coun- 


M 
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TABLE 7 
LENGTH OF HOSPITALIZATION 
Census I Total 
| 
Adm. Disch. 
Years Male Female Census I Census П ү betw. Бег, 
(N=998) (N=934) (N=1,932) (N=1,925) Pss cens. cens. 
481) N= SE 
(N=444) (N=211) 
% % % % % % % 
5 or less 7-4 3.8 5.7 6.7 л 28.8 26.5 
I 8.0 8.4 8.2 8.3 E 35.4 21.8 
2 9.5 8.4 9.0 5.8 х.$ 20.1 16.1 
3 527 3.2 4-5 5.2 5-4 47 7.1 
4 5-7 5-4 5.5 5.0 5.8 2.5 5.2 
5 Bod 4.6 4.0 4.1 4.9 1.4 3.3 
6-10 15-7 18.9 17-3 16.9 20.9 3-4 9.0 
11-15 20.1 15.5 17.9 15.6 19.9 1.4 7.1 
16-20 11.8 13.4 12.6 14.6 18.6 1.4 2.4 
21-25 5.4 7.2 6.3 7.9 10:1 Ж 1.4 
26-30 2.9 4.8 3.8 4.2 5.3 E 
31—35 | 2.9 2.4 2.6 2.8 3.6 2 

36-up I 3.2 2.3 2.6 3:4 

Undet. 8 шы S aS 
Mean 10.48 12.02 11.22 11.71 14.77 2:19 3.72 
Median 8.75 9.80 9.30 9.91 13.31 1.10 1.60 


tries.) The term “specialized education” includes 


college as well as the usual varieties of training 
Which, although not customarily given grade 
designations, have a more or less formal se- 
quence of tasks to be mastered. On-the-job 
training usually was not regarded as specialized 
education unless accompanied by formal class- 
Work. The designation “specialized education 
does not necessarily imply graduation from high 
school. Completion of the eighth grade is suffi- 
cient for inclusion, since it was not uncommon 
for an individual to have graduated from the 
cighth grade and then to have taken. one ог 
two years of training in a specialized subject such 


as agriculture. For the present generation gradua- 
tion from high school is generally a prerequi- 
site to further training. All those who have re- 
ceived any form of college, nurses' training. 
business school, etc., are grouped in the category 
"specialized education" and are not listed under 
any other category. 

The percentage of undetermined educations is 
relatively high and constitutes 2.4% of the hos- 
pital population (Census I= 10.7%), This may 
reflect the reluctance of relatives to state edu- 
cation when the patient has completed only a 
very few years of school or when they feel that 
his school performance has been unsatisfactory. 


TABLE 8 


EDUCATIONAL LEVEL 


" lation Census II Patient Sampl: 
Hospital Popu (corrected for ae 
Total Compared with Normal 
Census П E Xa Disch. Population 
erm. betw. betw. 
Level 1 Census I, Census Il “pop. кеп. сепз. Hospital Normal 
Male Fema! 5 (М =1,932) (А 21,925) (N =1,481) (N =444) (N 2211) | (N =240) (N =503,325) 
(N =991) (N =934 3 % % % % % 
% 
% 2 6. 9.5 19.7 7.0 
% T4 21.3 2259 ЫЧ 17.5 26.5 17.7 
9-4 grade 23.6 18.8 18:0 29-5 at 27.5 40.8 30.9 37.9 
5-7 grade 31:6 27.3 p 24-9 15 18.5 21.8 18.9 25.0 
oth grade 26.4 ER 13.9 aos i 6.6 m ga 
2 grade .6 17. 9-0 . * 10. 
пу spec jä ex 7.5 LH 955 4:3 8 1:6 
Huigation 4:4 295 10.7 2.4 £ 
et. 2.4 2. 


* College only. 
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The percentage of unknowns is also partially 
explained by the antiquity of some records. 

Comparison with the educational level of the 
general population presented a problem. U. S. 
Census estimates for 1947 indicate that 68%, of 
the population in the age-group 30 to 35 had 
completed more than cight school grades, con- 
tasted with only 28% age 65 and over. The 
disproportionately high percentage of older 
people tends to lower the educational level of the 
hospital population, necessitating the use of 
age-corrected data in making comparisons with 
the normal population. Table 8 provides data 
on the educational level of the age-corrected 
sample. 

In order to render the data comparable with 
the U. S. Census, the term "specialized educa- 
tion" in the age-corrected sample was redefined 
as college training only, and excludes business 
Schools, nurses' training, etc. Аз a result of this 
redefinition, the percentage in this classification 
declined from 5.5%, to 3.2%. Most of those ex- 
cluded are now in the group with g to 12 years 
of schooling. 


It is apparent that a high percentage of the 
hospital population has a poorer education 
than the normal population. Although the 
feebleminded in the hospital population un- 
doubtedly influence data on educational level, 
the percentage so diagnosed (8.6%) appears too 
small to materially alter the discussion. The 
achievement of the hospital population remains 
considerably below that of the normal popula- 
tion even after excluding from consideration all 


diagnoses of mental deficiency. Perhaps the Бах 
sonality handicaps of some patients. are reflecte 

in an inability to function efficiently or to 
remain at a task to completion (е.р., completing 
school). In our society low intelligence (and/or 
achievement) is usually associated with low ans 
come, as well as other social disadvantages, which 
tend to impose a psychological burden. Perhaps 
psychoses are associated with low intelligence 
in our socicty, or perhaps the more intelligent 
may, through their extra resources, or resources 
of friends and relatives, less often go to a state 
institution, and perhaps the more intelligent 
have a better prognosis and are less apt to re” 
main in a state institution. 


Diagnostic Classification 

Diagnoses given in this paper (Table 20) are 
based on the revised 1934 statistical class fication 
systems (4). The hospital population follows the 
national trend. Approximately Боб of the pa 
tients have a diagnosis of schizophrenia. A diag- 
nosis of mental deficiency is given 8.6% of m 
patients, versus an expectation, оп the ns g 
psychological tests, of a maximum of 2⁄4% 1n 
the general population, Of these mentally de- 
ficient patients, 32.39, are diagnosed as Being 
without psychosis and 67.7% are diagnosed aS 
with psychosis, The high percentage of patients 
diagnosed as without psychosis is partially due to 
the long waiting list at the state school for the 
feebleminded. On the basis of educational level 
most diagnoses of mcntal deficiency appear 
reasonably accurate, 


TABLE 9 
ADMISSION sumenda НЕ 1951 THROUGH 31 May 1932 
N= 

Code Diagnosis N Diagnosis N 
ooo Undiagnosed 5 17 Manic-depressive psychoses 36 
or Psy. with syph. meningo-enceph. 4 1731 Manic type 29 
02 Psy. with other syph. of CNS x 172 Depressed type is 
о; Psy. due to alcohol 2 174 Mixed type 4 
07 Psy. due to trauma 1 18 Schizophrenia таг 
o8 Psy. with cerebral arteriosc. 5I 180 Unclassified А 
o9 Psy. with disturb. of circul. 10 181 Simple type ag 
то Psy. due to сопу. disord. (epil.) 4 182 Hebephrenic type 2 
ır Senile psychoses 83 183 Catatonic type 2 
тїт Simple deterioration si 184 Paranoid type 39 
115 Paranoid type 15 185 Other types i^ 
12 Inv SI onal р ош 62 19 Paranoia and paranoid cond. 6 
та оа 39 20 Psy. with psychopath. personal. а 

A па type, Е 13 21 Psy. with mental deficiency : 
13 Sy. due to metabolic, etc., dis, H 22  Undiagnosed psychoses pr 
15 Psy. due to unknown or heredit. 23 Without mental disorder 44 
а550с. with organic change 8 230 Un 1, 551 d ^ 
16 | ychoneyroses 41 = 2 Alcohol inn n 

161 ysteria 9 n 

165 Reactive depression i 254 Mental achem lity 3 
166 Anxiety state 3 23 وا ی ی ا‎ 6 


237 Other non-psy. disease s 
24 Primary behavior disorder 


^ 
iu 
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Admission Diagnoses 


Table 9 includes diagnoses of all admissions? 
between 1 June 1951 and 1 June 1952 and per- 
mits comparisons with the frequency of diag- 
noses found among the patients in our census. 
The proportions of admission diagnoses of psy- 
choses with cerebral arterioscleros psychoses 
with other disturbances of circulation, senile 
psychoses, involutional psychoses, psychoneuroses, 
manic-depressive psychoses, and alcoholism are 
considerably higher than the proportions among 
patients included in the census. Conversely, the 
proportions of the diagnoses of psychoses due 
to unknown or hereditary cause but associated 
with organic change, schizophrenia, psychoses 
With mental deficiency, and certain disorders 
Without psychosis (excluding alcoholism) are 
higher among the census population. The above 
trends probably have some relationship to the 
prognoses of patients in these categories. 

Later discussion will take into consideration 
the shifts in diagnoses that have occurred be- 
tween the censuses, in an effort to determine the 
eflects of treatment, discharge, and death. 


Behavioral Ratings 


The appendix shows each of the areas rated 
on the scale. The number preceding each state: 
ment does not appear on the scale, Only a blank 
Space is provided for a check mark, Absolute 
interval equality is not claimed for statements 
that are made under each one of the areas rated, 
but it is felt that cach of the statements Tepre- 
sents a behavioral advance over the previous one. 
Needless to say, not all patients were rated on 
all areas, because some patients were not ob- 
Served in а particular form of behavior; also, 
Some ratings for an area were discarded as 1n- 
Valid, The average percentage of undetermined 
ratings on areas "A" through ug (excluding ag 
84.3%. 


Area “A,” Work 

The average female does à little work with а 
lot ot urging and needs constant supervision. 
On the other hand, the average male is much 
More apt to have a regularly 1 signed job under 
Supervision. (These statements are Бе und 
verbal approximations of the behavioral rati us 
And assume face validity of Scale statements) 
Chis difference between the sexes is both e 
Ustically reliable and clinically significant. There 
15 no reliable difference between the tW 
Censuses. 

It can be seen from Table 10 


in this paragraph an 
e-year perio 
d earlier 


that males are 


i "The term “admissions” 
ү: to admissions for this om 
Ot to "admissions" as define 
study, 


much more capable of functioning at the level 
of a rating between 3 and 5 points in the area 
"Work." A patient rated 4 is performing at a 
relatively normal level. In. most such cases, only 
a minimum of supervision is required, and the 
patient is able to make the routine decisions 
that the job necessitates. Twenty-one per cent of 
the men and 17% of the women perform at this 
level. 

An explanation of the superiority of males is 
difficult. Perhaps the majority of jobs in the hos- 
pital are of a type with which males are more 
familiar. Males are more frequently employed 
in a large establishment where functions such 
as farming, laundry, cafeteria, construction and 
maintenance, and janitor work closely approxi- 
mate those of the hospital. Much of the house- 
work of the institution bears little, if any, re- 
semblance to that done by the average house- 
wife, Dishwashing, for example, is a matter of 
perhaps 10,000 dishes per meal and is done on an 
assembly-line basis, Perhaps the male worker in 
the hospital enjoys greater prestige and receives 
more consistent rewards. Since the need for male 
workers is much greater, hospital industries may 
be more willing to devote time to training and 
keeping them. There are more open men’s wards 
(in which patients have freedom of the hospital 
grounds) then open women's wards. It is also 
possible that women patients as a group are 
somewhat more variable in their behavior. 


Area “B,” Response to Meals 


Males and females are approximately equal. 
The average patient seems to cat by himself 
using а knife, fork, and spoon properly, but may 
at times have some problem. It would also seem 
that the average patient will ask for things to 
be passed at the table but will do little else. 
Very few patients require special attention at 
meals. On Census I a total of only 159 patients, 
or 8% of the rated population, required this 
form of attention. There were no reliable 
changes between the first and second censuses. 


Area "C," Attitude to Other Patients 


Males are reliably better in this regard than 
females, On Census I the average female is 
somewhat unfriendly but may speak to others 
and even have a friend. The males, on the other 
hand, are more consistently rated as being 
friendly and having some spontaneity in making 
contacts and initiating play or work of a fairly 
high order. Approximately one-third of the pa- 
are quite seclusive and/or hostile. About 


tients E s 
are quite friendly and operate at а 


one-half 
fairly high social level. | 

Patient contact is of a higher order than is 
generally assumed. The greatest difference be- 


tween males and females is in the range 4 to 5. 
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TABLE 10 
RATINGS OF PATIENTS ON THE AREAS OF THE L-M SCALE 
Behavior Rating Mean 
Behavior Area and Group I 2 3 4 5 | Census Census Dis- 
% % pA % % I п charges 
A. Patient's attitude to work 
Male* 38.7 11.4 27.2 8.4 12.4 2.48 2.50 3.40 
Female 47.8 16.9 17.9 7.3 9-4] 2.20 2.14 3.21 
"Totale 43-1 14.1 22.7 7.8 mo 2.34 2.32 3.34 
B. Patient's attitude to meals 
Male 7.3 22.8 32.3 24.0 13.3 3.22 3.20 4.05 
Female 9.2 29.4 29.2 10.1 21.7 | 3.17 3.14 4.22 
Total 8.2 25.9 30.8 17.3 17.4 | 3.20 3-17 4-13 
С. Patient’s attitude to other ра- 
tients 
Male 31.7 14.1 32.9 11.0 8.7 2.56 2.65 3.07 
Female 36.6 17.6 32.9 5.9 6.6 2.35 -42 3-4? 
Total 34.1 15.8 32.9 8.5 7.7] 2.46 2.53 3.55 
D. Patient's attitude toward psychi- 
atric ae and nurses 
Male 20.5 26.7 24.8 15.5 8r 
^ ; 4.8 15.5 8.8] 2.74 2.87 3 
Female 22.0 35.4 18.3 8.5 14.0| 2.67 2.59 3.87 
Total 21.5 30.9 21.7 12.1 11.3 2.70 2.74 3.84 
E. Patient's attitude toward doctors, 
social workers, and psychologists 
Male 4.6 25.5 
т 5 41.7 13.1 5.2 | 2.94 2.90 3.01 
Female 10.3 32.3 40.7 7.2 1.1 2.62 PRO 3:39 
Total 7.4 28.8 41.2 10.2 3.2] 2.79 2.78 3:47 
Е. Patient’s response to electric or 
insulin therapy 
Male 1.0 i.3 
йыб та .6 | 2.90 2.90 341 
Female $5 GF 64 1.0 12 2.44 2.38 2.04 
Total 2:2 0 555 1.0 „| бо ma 2.88 
G. Patient's attitude to occupational 
and recreational therapy 
Male 34.1 17 
7 181 6.9 9.4| 2.36 2. 3.56 
Female 39.3 27.7 20.0 12.4 бу. s M 3.46 
tal 
Tota 37:2 22.5 зл 0:6 9.8 2.40 2458 3.52 
H. Attention to dress and person 
Male 9.3 33.8 1 
E a 3 4.3 19.2 14.6 ; .12 3.94 
Female 14.9 32.9 21.8 10.8 as 2. 2:83 4.00 
Total 120 38.3 7.9 їшї 16.4 | 2.97 2.08 3.97 
Note.— Because the rating for each area is the average of two raters, а patient may receive а rating 


that is not a whole number. (This occurs in approximatel 


113. 
„-М=д8, 
¢ Cen. I, N— 1,932; Cen. II, N=1,925; Disch., N=arr, 


— 


bas. 
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TABLE ro— (continued) 


Behavior Rating Mean 
Behavior Area and Group А а 3 Я 5 Gauls: Census, DE 
% % % % % I п charges 
I. Psychomotor activity 
Male 12.3 17.2 12.2 25-4 20.1 3.535 3.57 4.20 
Female 19.0 15.9 16.5 20.5 24.2 | 3.26 3.28 4.31 
Total 15.8 16.6 14.3 23.9 22.0 | 3.30 3.42 4.30 
J. Speech 
Male 20.3 20.7 8.5 18.5 28.0 | 3.22 3.18 4.34 
Female 18.8 14.0 14.8 22.5 26.0 3.35 3.12 4.47 
Total 19.6 17.5 11.5 20.5 27.0 3.28 Seis, 4.40 
К. Toilet behavior 
Male 9.2 4-3 12.9 15.7 56.5 4.10 4.16 4.70 
Female II4 7.6 4 o 39.1 | 3-73 3.91 4.58 
Total 10.3 5:9 17.5 15.8 48.1 3.92 4.04 4.65 


ce is not statistically relia- 
ble both males and females improved slightly 
between Census 1 and Census П. If this trend 
continues it may be a reflection of the increased 
emphasis on socialization. 


Although the differen 


Area “D,” Response to Psychiatric Aides 


and Nurses 

Males and females do not differ reliably. The 
average patient in this area docs mer ar 
When asked and may make simple requests: . i 
Census II males showed an improvement which 
approached statistical reliability- Females 45 
clined in this area. Patients are not as hose 
as one might assume. The results of ines 
social programming throughout the даш. "ie 
have paid greater dividends ОП te mas i 
of the hospital. This finding is consiste e 
the results of а total-push program for 
Bressed patients (1, 5)- 


Area "E, Response to Doctors; Social 
Workers, Psychologists 


nts rated in this 


The lower number ОЁ patie! 1 
vith non- 

um 7 e lack of contact W I 

Ше seston $ rsonnel in the hospital. 


nursin; rofessional ре! 

Again EE e a more often ар ada 
levels, A statistically reliab! c aifer ерге 
found on both censuses. though the 

thus far presented from 
that male patients show bette’ 
female patients, it must be reme 
the overlap is great. In this 2 
9% of the females have a nigher 


80% of the males and 5% of the males have a 
lower rating than 89% of the females. 


Area “Е Response to Electric or In- 
sulin Therapy 


This area concerns patient attitudes toward 
the administration of treatments, and only 
recently treated patients are rated. 251 patients 
were rated on Census I and 539 on Census II, On 
both censuses males were less antagonistic and 
apprehensive than females. There was no re- 
liable difference between Census I and Census 
II in the average response to this form of treat- 
ment. The larger N on the second census un- 
doubtedly represents an increased use of electric 
and insulin therapies in the hospital. 


xa “б? Occupational and Recrea- 
tional Therapy 


On Census I males and females did not differ 
reliably, but females tended to be higher. This 
may have been due to the effective occupa- 
tional and recreational therapy program that 
Fergus Falls has carried on for a long time, 
which was somewhat more extensive on the 
women’s side of the hospital, and which empha- 
sized things that women characteristically do, 
eg. embroidering, fine painting, etc. In Census 
II it was found that the males had shown a 
statistically rcliable improvement which made 
them equal to the females. The interim increase 
in recreational and occupational therapy pro- 
rams on the male wards and the initiation of an 
aide training program (resulting in greater ac- 
ceptance of, and more interest in activities) may 
explain patient improvement in this area. 
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TABLE :: 
AVERAGE OF AREAS "A" THROUGH "K''-"L'' 


Census I grouped by average rating | 


| Mean behavior rating 


Group 1.0-  2.0-  3.0- 4.0- 
1.9 2.9 3.9 4.9 
% 0 0 % 


5-9 | Census Census Perm. Admis- Dis- 


I II pop. sions charges 


Male! 17.1 30.8 32.0 20.1 
Female? 23.3 31.2 27.4 17.7 


2.96 3.28 3.90 
2.70 3.18 3.84 


Totale | 20.1 30.9 27.8 18.9 


® Cen. I, N=9ọ98; Cen. II, № —991; Perm. pop., 
^ Cen. І, N 2934; Cen. II, М —934; Perm. рор 


© Cen. I, N — 1,932; Cen. II, N — 1,925; Perm. pop. 


The average patient in occupational and 
recreational therapy will participate when asked, 
but will require frequent urging. He may show 
some spontaneity, 


Area “H,” Attention to Dress and Person 


Although the difference is not reliable, the 
males tend to be slightly better than the females. 
The average patient has some interest in his 
appearance. A comparison between the first and 
second censuses shows that the males improved, 
whereas the females tended to decline. The 
difference between males and females on Census 
II is reliable. Scale artifact тау be responsible 
for the somewhat atypical distribution of pa- 
tients in the various behavioral levels of this 
area. 


Area "I," Psychomotor Activity 

The average patient exhibits some purposeful 
behavior and also some behavior which 
result of the illness. The males showed a hi 
reliable increase in their behavior rating on the 
second census (gain = 2). At this level activity 
is usually purposeful although movements may 
be somewhat erratic in terms of hypo- or hyper- 
activity. (There are more female than male 
hyperactive wards in the hospital.) As a result of 
the improvement in male behavior the hospital 
population was reliably better on Census II, 


Area “J,” Speech 


There was no reliable difference in speech 
between males and females, although the females 
were slightly higher on the first census, The 
average in this area is quite high. Many of our 
patients speak in short, clear sentences if they 
have a request to make. There was a reliable 
decrease between Census I and Census П for 
the female population of the hospital (from 3-35 
10 3.12). The males remained constant. This is 
the first statistically reliable fluctuation that the 
females have shown in the present study. 


is a 


ighly 


; Disch., 
= 188; Disch 


Area “K,” Toilet Behavior 


As a group, patients are relatively high in 
terms of toilet behavior, With some exceptions 
such as being too neat or spending too much 
time or perhaps being incontinent on rare oc 
casions, the average patient is normal. Again 
the males are, on. the average, superior to the 
females, and the difference is large enough to be 
both reliable and clinically significant, Both 
males and females improved on Census II, and 
the female and total populations were reliably 
higher. This improvement may reflect the dis- 
continuance of the various physical restraints in 
the hospital, and the gradual retraining of pa- 
tients who had been untidy and incontinent for 
a number of years, 

It is generally conceded that women need more 
time and space in the bathroom in order to be 
“properly groomed.” It should be mentioned that 
toilet facilities in the hospital are more adequate 
for men. For example, wards of comparable size 
have the same number of stools but the men's 
wards also have urinals, 


ie Average of Behavior Ratings of Pa- 
tients 


An average of the ratings of a patient on the 
eleven behavioral areas is more reliable and 
Tepresentative of the patient's total behavior 
than any one area of the scale. Minor fluctua- 
tions and Variations will tend to be canceled 
out. The behavioral average of patients has been 
demonstrated to be both reliable and meaningful 
(3). Areas of the scale have not been studied so 
carefully. 

_The average for the entire hospital was slightly 
higher 9n Census II, but the difference was not 
Statistically reliable. Males were reliably better 
than females on both censuses. Males also showed 
a statistically reliable improvement between 
Census I and Census II, while females showed 
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TABLE 12 


AVERAGE BEHAVIOR RATINGS OF 50 STUDENT NURSES COMPARED 
WITH THE HOSPITAL POPULATION (CENSUS 1) 


| Агеа 
Group | 
А B € D G H I J K L 
Nurses | 4.6 4.7 4.0 4.4 4.2 4.9 4.8 4.8 4.9 4.6 
ients 2.3 3.2 2.5 2.7 2.4 3.0 3-3 3-3 3-9 2.0 


essentially no difference. The population that 
was in the hospital on both censuses (the “perma- 
nent" population) also showed a reliable differ- 
ence between males and females, the males being 
better, This very consistent tendency for males 
to be rated higher than females may be of con- 
siderable importance to the administration of a 
mental hospital. The permanent population 
shows no variation in average behavior ratings 
between Census I and Census П, although they 
had a lower behavioral average than those not 
rated on both censuses. 

The difference between aver 1 
ings for males and females, either admitted or 
discharged, indicates that females are approxi- 
mately .ı lower than males. But male admis. 
sions are not reliably higher, nor male discharges 
reliably lower in behavioral rating than female 
admissions and discharges: these facts, then, de 
not seem to account for the improvement shown 
by the males between Census I and Census Hn 

It was found in a follow-up of a group of pz 
tients treated on a total-push program (1, xí 
that males tended to improve more and hold 
their improvement. while females improved less 
and did not hold their improvement. The а 
gus Falls hospital population exhibits bes 
consistent tendency for mentally ill femal ات‎ 
have a lower average behavior rating than m 
tally ill males. 


These findings suggest а nec 
hospital. 


age behavior rat- 


са for more staff on 
Reorientation 
sard the prob- 
assistance 
or. Perhaps more 


such things а5 
jobs 


of the hospital admini 
lems of female patients ! 
in improving genera 
attention should be de 5 ” 
make-up, the fitting of clothing, the ty 
assigned to female patients and so on. 


h я und to 
The differences in pena ee ig e Be 
exis fi and Census 7 
t between Census І ime. Perhaps 


explained with certainty t this time. © 
the initiation and maintenance of an © 
therapeutic program (dating trom 
funds in 1949) resulted in а 
ment in the behavior of patients: 
sary to make further behavior? ıestions Te 
intervals to answer a number of q" 
garding the stability and areas of imp 
noted on these censuses. 


voted to 


Ratings on Student Nurses 


One might suppose that the L-M Scale would 
not differentiate adequately among normals 
since most of them would receive a rating of ap- 
proximately 5, which is the upper limit of the 
scale. Presumably, however, this average would 
give some indication of differences that may 
exist between “normals” and mental patients. 

A group of 50 student nurses were rated by 
two of their classmates on the L-M Scale. The 
students lived together in a nurses’ home for 
three months prior to the ratings, and had also 
worked on different wards in the hospital, The 
average rating that these students received was 
б. Four students received an average rating of 
4.2 and five received an average rating of 4.3. 
The two areas rated lowest were "C" (responsc 
to peers) and “G” (occupational and recrea- 
tional therapy). The rating was approximately 
that of the highest-level ward in the 
hospital. This ward has only long-term and 
convalescent patient workers (patients’ average 
was 4.6). It should be remembered that students 
are young, single females, of above-average in- 
telligence, who were rated on behavior in a less 
restricted environment than that of the patients. 

These results imply that more subtle things 
than the overt behaviors measured by the scale 
play а role in adjustment. Perhaps attitudes 
toward others or toward hospital living are 
very important. Certainly many patients at the 
level of 4:5 could leave the hospital if they chose 
to do so (and some do). Some patients at this 
level are unable to interact satisfactorily with 
other people and are not permitted to leave. 

The rankings of patients and students are 

iven in Table 12. The groups show the great- 
est differences on Areas "A" (work), | G (occu- 
pational and recreational therapy), “Н” (atten- 


F nd person), “D” (response to 


tion to dress anc T. 
diate supervisors), and "C" (response to 


equal to 


imme! 
eers). 
p Such things as present age, age at first ad- 
mission and so on were studied in relation to 


behavior rating of patients, and 
are quoted and discussed below. 
se noted, these analyses are from 


the average 
these results 
Unless otherwh 
Census I. 
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TABLE 13 TABLE 14 " 
ах RELA- 
GS IN RELATION TO AGE AND LENGTH OF HOSPITALIZATION IN REL: 
ая CONDITIONS TION TO DEBILITATING CONDITIONS 
Census 11 Non- Debilitated 
Item EU ebi 
Patient ima | debilitated | (y 204°) 
Census Sample (in years) (N = 1,6389] (N= 294' 
Status Population (corrected 

for age) Mean Present Age 52.10 63. 02 

Median Present Age 51.18 66.50 
Ме M N M 

Mean Age at ist Аат. 39.21 51.98 
Nondebilitated | 1,638 3.04 260 3.04 Median Age at 1st pe 

Debilitated 204 2.38 Sr 53.13 Adm. 35.81 БЕ 
Total 1,932 2.89 291 3.05 Mean Length of Hosp. 11.50 9.66 

Median Length of <8 
* Includes approximately .5% undetermined Hosp. 10.03 5.03 

cases. 


Behavior Ratings of the Physically De- 
bilitated 


There were 294 patients (150 male, 144 fe- 
male) who were considered to have physical 
handicaps. Since these handicaps would affect the 
ratings rather markedly, especially on such things 
as psychomotor activity, the authors excluded 
these people in the belief that such averages 
would be spuriously low. 

In the non-age-corrected data there is a 
marked difference in average behavior-rating be- 
tween those patients who are physically debili- 
tated and those who are not. This finding would 
be expected since many of the areas on the be- 
havior rating scale presume that the patient is 
physically able. The age-corrected sample of the 
hospital population was analyzed for average 
behavior rating of the physically debilitated. 
The exclusion of a large number of elderly pa- 
tients in the age-correction process resulted in a 
substantial improvement in the average rating of 
the physically debilitated group. In this age- 
corrected sample the behavior rating of the 
debilitated is equal to that of 
debilitated. 

On items with statistically reliable differences 
between the debilitated and nondebilitated, the 
results will be discussed for the two groups 
separately. 


the non- 


? Includes approximately .5% undetermined 


cases. 


Behavior Ratings in Relation to Present 
Age 

Nondebilitated patients between the ages of 
51 and 65 have an average behavior rating slightly 
higher than that of any other age group. Patients 
under 40 and over 75 tend to show poorer be- 
havior than the average patient. There is à 
modest curvilinear relationship (eta = а) be: 
tween present age and behavior rating. The 
physically debilitated population docs not exhibit 
a relationship between behavior rating and pres 
ent age. Aged patients are more frequently classi- 
fied as debilitated. The physically debilitated 
population is 10 years older than the total hos- 
pital population. In the Census II age-corrected 
sample, the physically debilitated patients are 
older (mean = 50.6) than the nondebilitated 
(mean — 43.5). sd 

Because of some differences in the definition 
of debilitation, caution should be exercised in а 
comparison of Census I and Census IJ data. 


Behavior Ratings in Relation to Age а! 
First Admission 

Table 15 indicates a curvilinear relationship 
between behavior rating and age at first ad- 


mission (eta — 27). Up to age 65, patients show 
à trend toward higher behavior rating as their 


TABLE 1$ 


AGE AND LENGTH ОЕ HOSPITALIZATION 
OF NONDEBILITA 


IN RELATION TO BEHAVIOR RATINGS 
TED PATIENTS 


Item | N r 7 (eta) PE. x n p 
Present Age 1,635 +.05 ET pe оот 
Ageatratadm | hes Уш Шо э ма по ш 
Length of Hosp. 1,630 —.13 19 516 ans 6 ооаб 


CENSUS OF A HOSPITAL POPULATION 17 


age at first admission increases. This increase in 
behavior rating is in accord with the hypothesis 
that the older an individual is upon becoming 
mentally ill, the less severe has been his degree of 
maladjustment in society. A priori, a patient who 
is admitted before the age of 25 would have 
more difficult adjustment than an individual 
admitted between 4o and 65. A person ad- 
mitted beyond the age of 65 has more difficulty 
E hospital adjustment than a somewhat younger 
patient. On age at first admission, the physically 
debilitated population is thirteen years older 
(mean — 52.0) than the nondebilitated popula- 
tion. (mean — 39.2). The relative. percentage o 
physically debilitated is much higher in the 
Older age groups. Physical deterioration (Te 
flected in a lower behavior rating) seems to play 
an important role in the admission of the aged 
patient, 


Behavior Ratings in Relation to Length 
of Hospitalization 

Patients hospitalized two years OF ty? 
definitely higher behavior rating than other pa: 
tients. There is a steady regression ird 
rating up to 20 years of hospitalization. After 
this time the ratings begin tO | 
those patients who have been hospit 
years show behavior that is equal t 
than the average patient in the hospital. After 
45 years there is a very marked regression 1 
behavior, probably as a result of aging. тнеш 
interesting aspect of this relationship is the defi- 
Nite cessation of regression that occurs. This 
EBES that behavioral regression js not con- 
inuously progressive. This hypothesis 18 ef 
Ported by the highly reliable difference between 
the correlation ratio and the product-moment 
correlation (cf, Table 15). i 

Obviously, the hypothesis of continuing re- 
Bression is inadequate. The hospital's experience 
With its older long-term patients nas indicated 
that many show spontaneous remissions: ей 
illnesses are frequently said to have wd 
Out." The needs of an individual of 59 

T 


ber of aged patients could b E 1 
were not so depesitent upon the hospita within 
кы institution they аге capable of fu 
a very high level. 
ye his phenomenon of i 
ars of hospitalization emphasizes 
Portance of studying the long-term MEP cents 
€neralizations made from short-term. FEN 
nn not necessarily applicable to all Л! 
CSS. ian 
Physically debilitated patients have à P 
*ngth of hospitalization approximate? op. 
Years less than that of the nondebilitatec Б i 
Ulation, The median for debilitate 


improvement after, 
imp the im- 


TABLE 16 
BEHAVIOR RATINGS IN RELATION TO RELIGION 
Religion N | м 
Protestant 1,008 3-07 
Catholic 404 3.02 
Other 26 3.00 
No Religion 144 2.96 
Total | 1,582 | 


approximately four years and the median for 
females is approximately eight years. The gen- 
eral trend is consistent with that of the total 
hospital population, but the physically de- 
bilitated patients are older and their length of 
hospitalization less, probably because of a high 
mortality rate. The hypothesis that there is 
higher mortality among the physically debili- 
tated, especially among males, is supported by 
their older age at first admission, and their very 
short length of hospitalization. If the physically 
debilitated were included, the improvement 
shown by patients after 20 years of hospitaliza- 
tion would be cven more marked because of 
the lower behavior rating of physically debili- 
tated patients and the shorter length of their 
hospitalization. . 

Age at first admission and length of hospitali- 
zation are inversely related. The relationship of 

resent аре to these two items should also be 
considered in interpreting census data. For 
example, the relation of present age to behavior 
rating is undoubtedly influenced by both the age 
at first admission and the length of hospitaliza- 
tion. Though present age may be the same, the 
who was the younger at first. admis- 
1 ill tend to have the longer hospitalization 
ower behavior rating. 


Behavior Ratings in Relation to Reli- 


gion : 

There is nO relationship between religious 
affiliation and mean behavior ratings. Patients 
who belong to minor religions or who have no 
religion do not differ markedly from those who 
belong tO one of the major religious sects. In 
the normal population members of minority 
sects and those who profess no religion are fre- 

uently considered to be maladjusted. 

physically debilitated patients have essentially 
the same proportion of religious faiths as the 
nondebilitated population. 


Behavior Ratings in Relation to Marital 


Status 

The lower rating of widowed patients may 
reflect. their advanced age. Single patients arc 
wer in behavior rating than those who have 


lo d 8 = 
been married or divorced. Divorced patients have 
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The age-corrected sample shows essentially simi- 
lar characteristics. 


Behavior Ratings in Relation to Diag- 
nosis 


This area presented a number of problems. 
The bias of the individual responsible for final 
diagnosis, the orientation of the institution, and 
the difficulty of sorting patients into Kraepelin- 
ian diagnostic categories are all responsible for 
some confusion in diagnosis. Older diagnoses 
may also be inaccurate, although most of the 
more obvious errors in diagnosis in this hospital 
have been changed. The major role of diagnosis 
in this institution is probably that of assistance 
in determining the treatment to be given a pa- 
tient. А 

Table зо is based upon the 1934 classification 


of mental disorder. Diagnoses in Census II have 


been fitted to this classification (4). Because of 
een the male 


occasional marked differences betweer 
and female populations, characteristics of both 
are given. There are some differences 1n diagno- 
ses between the two censuses. A gradual decline 
can be noted in the number of patients with a 
diagnosis of psychoses with syphilitic meningo- 
encephalitis (general paresis). Behavior ratings 


of the patients with this diagnosis have bed 
become lower. More adequate treatment methods 
ay account for 


(both preventive and arrestive) m 
the declining numbers. 
The number of pati 


ents diagnosed as psycho: 
ses due to alcohol has also shown a very marked 
decline. Behavior rating of patients with this 
diagnosis has become higher. Admissions i me 
new state hospital for alcoholics changed th 
admission rate at this hospital. Perhaps carlict 
and better treatment of alcoholics prevents a 
number of these psychoses from developing. Ай 
Under the diagnostic heading psychoses w 3 h 
cerebral arteriosclerosis,” approximately и 
thirds of the patients are male. The men ре 
havior rating approximates the average pen 
hospital population. In the dag 
enile psychoses " the number О р a 
creased markedly between Census I and Cens 


II. This increase was especially noticeable in the 
diagnostic category "senile psychoses with simple 
deterioration.” The very low average behavior 
rating of this diagnostic category suggests the 
possibility that the differential diagnosis between 
senile psychosis and psychosis with arteriosclero- 
sis is made on the basis of age and lower be- 
havioral level. (It should be noted that the death 
rate for the diagnosis “senile psychoses with 
simple deterioration” is much higher than that 
of the other groups.) The diagnosis of senile 
psychoses, paranoid type, has a behavior rating 
approximating the hospital average. Many more 
females are diagnosed as having involutional 
psychoses, The behavioral rating of patients with 
this diagnosis is considerably above that for the 
hospital population. 

On Census II the senile and arteriosclerotic 
diagnostic categories show а definite increase in 
the absolute number of females. If females have 
a longer life span, and if the trend for increased 
senile admissions continues, опе might anticipate 
that in the future there will be more female 
than male patients in the hospital. 


Ages on Admission in Relation to Diag- 
noses 


Mean ages of patients admitted between 1 June 
195! and 1 June 1952 in the categories “psy- 
choses with cerebral arteriosclerosis,” “senile 
psychoses,” and "involutional psychoses” give 
further indication that these diagnostic differen- 
tiations may be made, at least partially, on the 
basis of age and behavior, The age differences 
between the three diagnostic categories are all 
significant. The mean age of male and female 
patients is approximately equal in cach of the 
categories. 

On the basis of some theories of the involu- 
tional period it would be anticipated that males 
diagnosed as involutionals would be about ten 
years older than females. Table 21 indicates that 
this is not the case. Further, males receive an 
involutional diagnosis infrequently. Apparently 
there is a strong male-female bias existing be- 
tween the diagnostic categories “psychoses with 


TABLE 21 


x TO DIAGNOSES 1 Јох 


E 1051 THROUGH 31 МАҮ 1952 


AGEs ON ADMISSION IN RELATIO = ——— 
- per وع‎ Male | Female 
Code 1 yiagnosis № Аве SD | С Ме i 
NET as pius 8.8 | 10 71.0 7-5 
о8о | Psy. with cerebral агїег1о5©. 5 id pe 2 ü d 23 
A Senile ph. deterioration sd 53-4 бз | Ж 81.6 9:8 
E. ia | Involutionol psychoses te and one female patient. 
H 


one 
^ The age was unknown for 
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cerebral arteriosclerosis” and "involutional psy- 
choses." Extremely old patients are diagnosed 
as senile psychoses with simple dementia; 
younger male patients are diagnosed as psychoses 
with cerebral arteriosclerosis; and younger fe- 
male patients are diagnosed as involutional psy- 
choses. In view of gross differences in age and 
behavioral level in the categories, the possibility 
that diagnosis is not entirely on a physiological 
basis should be considered. 

The diagnostic group “psychoneuroses” (Table 
20) has a total of 43 patients. The mean be- 
havior rating of these patients is considerably 
above that of the hospital as a whole. The most 
common subgroup diagnoses were anxiety-hys- 
teria, reactive depression, and anxiety state, The 
psychoneurotics that are received in a state hos- 
pital may be much more maladjusted than those 
commonly encountered in private practice, since 
it is usually assumed that a neurotic is capable 
of adjusting within the community. 

The group diagnosed as manic-depressive psy- 
choses has a preponderance of female patients. 
Males are rated much higher than females, and 
the group rating is considerably above the hos- 
pital average. The behavior rating of those di- 
agnosed as manic has shown a reliable decrease 
between Census I and Census II. 

A majority of the patient population is in- 
cluded under the diagnosis "schizophrenia." On 
Census I, 968 patients had this diagnosis and 
889 of these were in the hospital at the time of 
Census II. Only 74 patients with this diagnosis 
were discharged. This rate is below that ex- 
pected on a statistical basis, Many more males 
than females received the diagnosis "simple 
schizophrenia." It will be noticed that the be- 
havior rating of this group approximates that of 
the hospital. Of the 257 patients who had a 
diagnosis of schizophrenia, hebephrenic type, 
only four were discharged. Even though the 
admission rate is low, the discharge figure seems 
small for the relatively large number of these 
patients, (It is possible that this diagnosis was 
given more frequently in the past and current 
figures are not comparable.) The behavior rating 
of the hebephrenic patients was below that of 
the hospital. A diagnosis of schizophrenia, cata- 
tonic type, was given to 82 patients. The be- 
havior rating for this group is also considerably 
below that of the hospital. On the other hand, 
the number of patients discharged (N — 11) from 
this group was fairly high considering the low 
number of admissions. The impression that pa- 
tients diagnosed as catatonic schizophrenia have 

a fair prognosis agrees with views of others. The 
majority of patients diagnosed "schizophrenia" 
were of the paranoid type (407 patients). In this 
hospital this diagnosis is used as somewhat of a 
“catch-all.” If a patient does not fit too well 
into another schizophrenic classification he is 


usually diagnosed as a paranoid type. The aver- 
age behavior rating of the paranoid schizo- 
phrenic is slightly higher than the hospital 
mean. There were a large number of patients in 
the undifferentiated classifications (c.g., such di- 
agnoses as dementia praecox) and many of these 
were reclassified in the interval between censuses. 

Many patients are diagnosed “psychoses with 
mental deficiency.” The behavior rating of this 
group approximates the hospital average. The 
data indicate that a dual handicap of psychosis 
and mental deficiency does not prevent release 
of some patients to a noninstitutional environ- 
ment, " 

In the category "without mental disorder, 
the patients diagnosed as mentally deficient form 
the largest subgroup. The behavior rating of 
this subgroup approximates the hospital mean. 
The alcoholics without mental disorder have thc 
highest behavior rating in the hospital. A great 
many in this group were discharged. The psy- 
chopaths in this hospital also had a very high 
behavior rating. Approximately onc-half of the 
individuals in this group were rediagnosed in 
the interval between censuses. 

Most of the conclusions that can be rcached 
from the diagnostic groups seem similar to those 
previously reported. If the data are adequate, 
some of the generally accepted concepts of Te- 
gression (especially those regarding the diagnostic 
category “simple schizophrenia”) need a rc 
evaluation, Since hospitals differ in the diagnoses 
that are attached to various behavior patterns, 


one would expect differences in inter-hospital 
comparisons. 


Behavior Ratings and Regression 


An interesting speculative possibility concerns 
the relationship between the levels of behaviors 
rated on the behavioral census and those hy- 
pothesized in psychological theory. It has been 
widely held that latest learned behavior shows 
maximal regression and that earliest learned be- 
havior shows minimal regression. Four psycholo- 
gists and one psychiatrist from the University 
of Minnesota Hospital Psychiatric Unit were 
asked to rank independently the areas in the 
behavior rating scale in order, from earliest tO 
latest learned behavior, (These clinicians did not 
know the results of the behavioral census.) The 
areas on the behavioral rating scale were also 
ranked from highest to lowest in terms of their 
mean values on Census I. 

Agreements were computed by the rank-difler- 
ence correlation method. The rankings of each 
clinician were compared with the rankings of 
each other clinician and also with the rankings 
determined from ‘Census I. The ranks of each 
clinician were averaged for each individual area 
and a correlation was computed between these 
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TABLE 22 


CORRELATIONS (RHO) BETWEEN Five CLINICIANS’ 


RANKINGS OF BEHAVIOR AREAS AS TO EARLY VS. 


AREA; AND CORRELATION (RHO) BETWEEN CLINICIANS' 


Late LEARNING OF THE : 
RANKINGS AND AVERAGE BEHAVIOR RATING FOR EACH AREA IN Census І 
Clinician 
Clinician Average of 5 
I ll lll IV v Clinicians 
1 
IL .69 
111 .63 .8o 
IV E -79 -76 
V .61 .89 .90 .89 
Rank based on mean behavior 
rating of areas, Census Р .67 .84 .68 .66 .69 .8o 


were r: 


^ High behavior ratings 1 
and thus yiel 


vide a measure of regression 
areas with respect to early vs- 


а positive 


average ranks and the ranks from the behavioral 


census, The correlation by this method was .80. 
The ranking of the areas on Census I from high 
to low is as follows: K. Toilet Behavior; I, Psy- 
chomotor Activity; J. Speech; B, Response to 
Meals; H, Attention to Dress and Person; E. 
Response to Doctors, Social Workers, Psycholo- 
gists; D, Response to Psychiatric Aides and 


Nurses; F, Response to Electric or Insulin Ther- 
1 Recreational Therapy: 


ару; G, Occupational anc 

C, Response to Other Patients; A, Work. The 
ranking of the arcas by the clinicians, with re- 
spect to early vs. late learning of the behavior 
area, is: B, Response to Meals; K, Toilet Be- 
havior; I, Psychomotor Activity; J, Speech; H, 
Attention to Dress and Person; C, Response to 
Other Patients; E, Response to Doctors, Social 
Workers, Psychologists; D, Response to Psychi- 
atric Aides and Nurses; A, Work; G, Occupa- 
F, Response 


anked low, and low behavior rati 


ngs were ranked high, so as to pro- 


correlations with the clinicians’ rank-order of the 


late learning of the behavior area. 


correlation is not high enough to be of predictive 
value, it indicates a definite relationship between 
psychological theory and the pattern of regres- 
sion exhibited behaviorally. 


Intercorrelation of Areas 


The intercorrelations of the areas on the be- 
havioral census were computed, With the ex- 
ception of Area F, Attitude toward Insulin or 
Electric Therapy, the correlations of areas of 
the scale with the average (L) ranged from .69 
to 87. The best predictor of the average rating 
was Area D, Response to Psychiatric Aides and 
Nurses. Area F showed relatively little correla- 
tion with other scales, the correlations ranging 
from .об to .36. The other scales (excluding F) 
show intercorrelations of .42 to .78 with the most 
typical correlation being about „бо. A factor 
analysis of this matrix found only one factor. 

These correlations show а moderately high 


tional and Recreational Therapy; 
to Electric or Insulin Therapy. Although the positive relationship to the general average and 
TABLE 23 
INTERCORRELATIONS OF AREAS ON THE L-M SCALE (FROM CENSUS D 
pn A B С р E F E H E J K 
» سےا‎ — 
А 
B .66 8 
.67 ‚01 
D 78 .69 .16 63 
E ee “ыз 36 
Е о NE - Ше ay cda 7 
G .63 -54 01 .66 .50 3 S бе 
н .60 +72 65 .68 .50 ES ET 65 © 
] ei | oe | DEM ш OE, 
ЮЕ ЛЕ e 
.83 . = 
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to each other. The fact that Response to Psychi- 
atric Aides and Nurses (Area D) has the highest 
relationship to average behavior rating implies 
the operation of “halo” effect. Several factors 
should be considered in assessing the importance 
of this bias. The complexity of interpersonal re- 
lationships may mean that “less normal” patients 
are in fact lower in this area. “More normal” or 
“less normal” behavior may be a reflection of 
the patient's like or dislike for an aide (e.g., a 
feeding problem is more work for the aide). 
The relatively modest intercorrelations of the 
areas (especially Area F) suggest that “halo” 
effect is probably minimal. In this connection it 
is often found, for example, that a patient will 
be a good worker and eat well, but will be poor 
at socializing with his peers. A wide variety of 
behavioral configurations can be observed. 


SUMMARY 


Two evaluations were made of the Fer- 
gus Falls State Hospital population, one 
in 1951 (N = 1,932), and one in 1953 
(N = 1,925). (The 1951 survey is referred 
to as Census I, and the 1953 survey as 
Census IL) Data were gathered on the 
following items: Diagnosis, Present Age, 
Sex, Marital Status, Religious Affiliation, 
Physically Debilitating Conditions, 
Length of Hospitalization, Education, 
and Age at First Admission; and on the 
areas of the L-M Fergus Falls Behavior 
Rating Scale. Some characteristics of the 
normal population as given in the 0. S. 
Census figures for the State of Minnesota 
[or 1950 were used as a basis for compari- 
son. These items were studied in an at- 
tempt to describe behavioral characteris- 
tics of a state hospital population. A de- 
scription of this type has not previously 
been attempted on a state hospital pop- 
ulation. It is hoped that results of this 
census will be of value to scientists, hos- 
pital administrators, and ultimately the 
lay public. 

Conclusions reached from the stud 
follow: 

1. The hospital population is older 
than the normal population, The aged 
in the hospital population are increas- 


y 


ing. Hospital planning must take these 
facts into account. 

2. Patients included in the census were 
most frequently admitted in early adult 
life (ages 20 to до). However, a group of 
consecutive first admissions (from 1 June 
1951 through gı May 1952) had a mean 
admission age of 53.9 years. This figure 
agrees with a recent figure for all Minne- 
sola state mental hospitals. 

3. А majority of patients and a minor- 
ity of normals are single. A correction for 
age greatly increases this difference. 

4. There are more widowed people in 
the hospital than in the normal popula- 
tion. However, an age-corrected sample 
did not differ reliably from the normal 
population in this respect. 

5. There probably is no difference be- 
tween the hospital and normal popula- 
tions with regard to religious preference. 

6. There may be more physically de- 
bilitated people in the hospital than in 
the community. Advancing age may re- 
sult in unfavorable personality changes 
as well as physical debilitation. 

7. The mean length of hospitalization 
was 11.9 years, Therapeutic program- 
ming should take long-term patients into 
account. Females are hospitalized longer 
than males, 

8. Even when an age-correction is 
made, hospital patients have less educa- 
tion than the normal population. 

9. The hospital has a larger number 
of people diagnosed as feebleminded 
(8.6%) than would be found in a normal 
Population on the basis of intelligence 
test distributions. 

10. The percentages of the various di- 
agnoses in this hospital agree substan- 
tially with those found by others. А ma- 
jority of the patient population is diag- 
nosed as schizophrenic. 

11. A comparison between diagnoses 


m 


4 


y 
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of admissions (for a one-year period) and 
the hospital-census population indicates 
that the two groups differ markedly in 
some categories. The admissions are 
more frequently diagnosed as psychoses 
with circulatory disturbances, senile psy- 
choses, involutional psychoses, psycho- 
neuroses, manic-depressive psychoses, and 
alcoholism. The hospitalcensus popu- 
lations are more frequently diagnosed as 
psychoses due to unknown or hereditary 
causes but associated with organic 
change, schizophrenia, psychoses with 
mental deficiency, and certain categories 
without mental disorder excluding alco- 
holism. 

12. In regard to the general level of 
work performed in the hospital, males 
are rated as being reliably better than 
females. The average male is capable of 
holding a regularly assigned job under 
supervision. The average female seems 
to do a little work, with much urging, 
under constant supervision. Perhaps 
males are more accustomed to working 
in an industrial setting. | 

13. Males and females аге approxi- 
mately equal in their behavior at meals. 
The average patient is usually able to 
eat by himself, using utensils properly. 
Very few patients require special atten- 
tion at meals. 

14. Males are reliably better than fe- 
males in social contacts with peers. Many 
males exhibit some spontaneity in mak- 
ing contacts and in initiating play or 
work of a fairly high order. The average 


female patient may be somewhat friendly 
ave an occasional friend. 


and may even h 
s and females re- 


15. On Census I male rales 
spond equally well to psychiatric aides 
and nurses. On Census II the average 
male has shown improvement. He has 
y better than the average 


become reliabl 
apt to make 


female and is much more 


simple requests and to do things when 
asked. 

16. Males are more responsive and 
friendly to non-nursing professional per- 
sonnel than females. 

17. Males are less antagonistic than 
females in the acceptance of the admini- 
stration of electric or insulin therapy. 

18. On Census I females tended to be 
better than males in response to occu- 
pational and recreational therapy. On 
Census II the male ratings show a statis- 
tically reliable improvement and become 
equal to the females. The average pa- 
tient in occupational and recreational 
therapy will participate when asked and 
may occasionally show spontaneity. 

19. Males show a tendency to be better 
than females in their attention to dress 
and person. The males improved at the 
time of Census II while the females were 
rated slightly lower (the difference be- 
tween them became statistically reliable). 
The average patient has some interest 
in his appearance. 

оо. Males and females are approxi- 
mately equal in level of psychomotor 
activity. Males showed a highly reliable 
increase in the level of their behavior on 
the second census. There are more fe- 
male than male hyperactive wards in the 
hospital. The average patient seems to 
exhibit mainly purposeful behavior but 
has some erratic movements which are a 


result of the ilIness. 

21, Males and females are approxi- 
mately equal in speech, although females 
were somewhat higher on the first cen- 
sus. Between Census I and Census II 
the females showed a statistically relia- 
ble decrease in this area. This is the first 
reliable shift reported for females in this 
study. The average patient will speak in 
short, clear sentences if he has a request 


to make. 
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22. 'The males are reliably better than 
the females in their toilet behavior. This 
difference is of clinical significance. Both 
males and females improved on the sec- 
ond census. The total improvement was 
statistically reliable. The average be- 
havior of the patient is normal except 
for being too neat or spending too much 
time at one thing and perhaps being 
rarely incontinent, 

23. The males were reliably better 
than the females on both censuses on the 
average of all these areas of behavior. 
The authors feel that this difference is 
large enough to be of clinical import- 
ance. The males showed a reliable im- 
provement between Census I and Census 
П. Females remained the same. Physi- 
cally debilitated male and female pa- 
tients have equal behavior averages. 

24. Male admissions and discharges are 
higher in behavior rating than female 
admissions and discharges. 

25. The consistent tendency for female 
patients to exhibit a lower level of be- 
havior than male patients (and the re- 
sults of a previous study in which fe- 
males failed to hold their improvement 
on a total-push program) (1, 5) would in- 
dicate the need for more staff on the 
women's side of a state hospital. State 
hospital administrations might profitably 
reconsider the problems of the female 
patient with regard to such things as 
type of work, available clothing, and 
toilet space. 

26. It is possible that an expanded 
therapeutic program was responsible for 
the improvement in behavior shown by 
the hospital population between Census 
I and Census II. Further censuses will be 
needed to substantiate this conclusion. 

27. The average behavior rating of a 
group of student nurses (“normals”) was 
approximately equal to that of the high- 
est-level ward in the hospital. Some pa- 


tients at this high behavioral level could 
leave the hospital if they were moti- 
vated to do so. 

28. Patients who are physically debili- 
tated have a lower behavior rating and 
are ten years older on the average than 
those who are not physically debilitated. 
The behavior rating of debilitated pa 
tients is fairly constant for all age groups- 

29. Behavior of patients between the 
ages of 51 and 65 is slightly better than 
that of any other age group. Those pa 
tients over 86 show poorer behavior. 
Otherwise no definite relationship be- 
tween age and behavior rating was dis- 
cernible. 

30. Up to age 65 patients show a trend 
toward higher behavior rating as age at 
first admission increases. Admissions after 
age 65 are poorer in terms of behavior. 
The physically debilitated population 15 
ten years older than the total hospital 
population on age at first admission. І 

31. Up to twenty years of hospitaliza- 
tion, patients show a steady regression 
in behavior rating. Those who have been 
hospitalized between 26 and 45 years 
have a behavior rating higher than the 
average patient. Inclusion of physically 
debilitated patients would make the 
trend more pronounced. After 45 years of 
hospitalization there is a marked regres- 
sion in behavioral rating, which may be 
due to aging. The hypothesis of continu 
ing regression should be re-examined. 

32. Type of religious affiliation (OF 
lack thereof) does not seem to affect be- 
havior rating. 

33. Patients who have been divorced 
have the highest (most favorable) Þe- 
havior rating; patients who have bee? 
married have the next highest; ап 
those who are single tend to have the 
lowest rating. Single individuals may be 
more severely maladjusted than thosë 
who marry. As a group, physically debili- 
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tated patients are rated lower than the 
nondebilitated. Both groups have ap- 
proximately the same relative ranking. 

34. Behavior rating has a positive re- 
lationship to the number of years of 
school completed. 

35. Because of the ambiguities that 
continually plague nosology, as well as 
the variations in biosocial environs, diag- 
noses present problems in intra- and 
inter-hospital comparison. More females 
than males receive manic- depressive diag- 
noses. The number of females has dis- 
proportionately increased between cen- 
suses in the categories of senile and in- 
volutional psychoses. If the trend toward 
older admissions continues, the longer 
life span of the female may eventually 
result in the presence of more female 
than male patients in the hospital. It is 
suggested that the differentiation be- 
tween the categories of psychoses with 
cerebral arteriosclerosis, senile psychoses, 
and involutional psychoses is not entirely 
on a metabolic basis, but is, at least par- 
tially, a. function of age, Sex, and be- 
havioral level. 


36. The mean behavior ratings of 
simple schizophrenics approximate the 
average for the hospital. The average 
rating of hebephrenics was below the 
hospital mean. The catatonics’ average 
was below the hospital mean but the rate 
of discharge was fairly high. Paranoid 
schizophrenics have an average rating 
slightly above the hospital mean. 

зт. Mentally deficient patients (with 
or without psychosis) are about average 
in behavior rating. Alcoholics, psycho- 
neurotics, and psychopaths are very high 
in rated behavior. 

38. A comparison of clinicians’ rank- 
ings and the results of the behavioral 
census lends support to the hypothesis 
that regression in mental illness occurs 
from latest to earliest learned behavior. 

gg. The most typical intercorrelation 
of the ratings on the behavior areas was 
about .бо (all correlations were positive). 

Although behaviors tend to be positively 
interrelated, this finding supports the 
observation that patients may function 
at different levels in various areas. Factor 
analysis yielded only one factor. 
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APPENDIX 


L-M Fercus FALLs BEHAVIOR RATING SHEET (2, 3) 


Name Hospital Number Age Sex Marital Status Date 
Diagnosis Time on Present Ward Ward Religious Preference 
i i ‘thi T d. 
—doesn't do anything requeste! 
A. Work. ing) : hing | 
(1) Does no work—refuses—extremely nega- — Will do a few things if asked or pushed 
tivistic, 


(2) Does a little work with a lot of urging. 
Constant supervision is necessary. 

(3) May have a regularly assigned job—and 
supervision may be necessary. 

(4) Enthusiastic participation in all types 
of work—asks for work. 

(5) Normal interest in work—i.e., interested 
in some kinds of work more than others 
(will do other kinds than main interest 
if called upon to do so). 

(If works, at what, and how many hours a day?) 


B. Response to Meals. 


——Has to have special attention, as cats 
too much, or is spoon-fed or tube-fed. 

— Eats by self, is sloppy—may need coaxing. 

——Eats by self using knife, fork, and spoon 
properly—may show some finickyness. 

———Passes and asks for things to be passed, 
but will not carry on table conversation. 

——Would not stand out among, normal 
people for eating habits. 


C. Response to Other Patients. 
—Stays all alone or may strike out at 
other patients, 
—— Will be with other patients only for a 
short while and with urging. 
Some signs of friendliness—speaks to pa- 
tients—may have a friend. 
Some spontaneity in making contacts 
with other patients. May initiate play or 
work of a social, relatively high type, 
e.g., card games, washing dishes, etc. 
——Helpfulness expressed toward other pa- 
tients—or non-hostile recognition of their 


being mentally ill and making allow- 
ances. 


(If stays alone or strikes, which one? 


D. Response to Psychiatric Aides and Nurses, 
Negativistic—hostile (can include strik- 


" The number preceding each Statement does 
not appear on the scale. Only a blank Space is 
provided for a check mark, A clerk assigns the 
numerical value. 


—shows no open hostility. 3 

——Will do most things when asked—will 
ask for simple things—"I want my tooth- 
brush.” В 

tremely cooperative—will do anything 
when asked. ' : 

—— Normal give-and-take relations цр. 
Speaks spontaneously to nurses abou 
things of not immediate importance, 
e.g. weather, baseball games, etc. 


. Response to Doctors, Social Workers, Psy 


chologists. 

— —Hostile. — 

——Passively negativistic (would rather па 
have anything to do with them but wi 
not resist), 


— Will speak when spoken to. 
—Seeks advice, 


er- 
——Understands, accepts, and asks for th 


apy. 
. Response to Electric or Insulin Therapy- 
— Hostile. rly 
Anxious, apprehensive, but not ove 
hostile. 


——Passively accepts. өт feel 

— Accepts positively—(may say, I 
better after”), 

—Asks for, understands necessity for. 


| 
. Occupational Therapy and Recreation? 


"Therapy (walks don't count). 


ativistic 
Does not participate at all—negativ! 


—hostile. hort 
Participates with urging for 5 
periods. 7 
РЭН апе 
Participates when asked—some spont 
ity. 


— Shows interest—participates in all YR 
wholeheartedly without discriminati 8 
very much between different types—loo™ 
forward to. — 

—— Interested. in many varied activi jt 
normal selectivity (likes some kinds mo 
than others). 


(What is patient most interested in?)—— 


» m 
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H. 


at 


CENSUS OF A HOSPITAL POPULATION 31 


Attention to Dress and Person. 

Has to be dressed—needs special atten- 
tion of one kind or another. 

— —Dresses self but is sloppy- 

— Some interest in looks—too much lip- 
stick, fairly neat. 

Cares about looks and dress—will ask for 
make-up or shaving equipment incon- 
sistently (not an over-all balance). 
Normal (for culture)—would not stand 
out in a crowd. 


. Psychomotor Activity (not including going 


to the bathroom or meals). 
— Stays in one place unless pushed—or hy- 
peractive (e.g. seclusion necessary). 
Moves around а little (one chair to 
another) or, if hyperactive, the activity 
is not of a type making seclusion or 
other restrictions necessary. А 
— Some activity resulting from the influ- 
ence of the illness (moves around be- 
cause voices say to) and some purposeful 
behavior. » 
Behavior mainly purposeful. Still moves 
around a little fast or a little slow. 
— Normal activity—would not stand out 
among normal people. 
hyperactive or stays in one place, 


which one?) 


ю 


‚ Speech. 


. Lucero, R. J, & MEYER, B. 1 


. Meyer, B. T, & Lucero, К. J 


. NATIONAL COMMITTEE FO! 


— Mute or speaks а lot but doesn't make 


sense. 
— —A few words that m: 


"no"). 


ake sense ("yes" or 


— Speaks in short, clear sentences—“Can I 
have my toothbrush.” 

—— Speaks normally except a little fast or 
slow. 

— —Speaks normally. 


(If mute or senseless talk, which one?) 


. Toilet Behavior. 

Untidy anytime during the day and/or 

more than twice a week nightly. 

— —Untidy once or twice a week nightly— 
brushes teeth and washes only when told 
to do so. 

— —Not untidy—toilet behavior somewhat 
sloppy—brushes teeth and washes once 
a day without being told. 

— — Toilet behavior normal except for being 
too neat, or spending too much time at 
one thing, or occasionally sloppy. 

— — Toilet behavior normal. 


Li 


List below anything physically wrong with pa- 


tients arms or legs. 


List below anything physically wrong with pa- 
tient’s hearing, sight, and speech organs. 


List below any physical illness that patient has. 
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The Retinal Size of a Familiar Object as a 
Determiner of Apparent Distance’ 


Walter C. Gogel, Bryce O. Hartman and George S. Harker 
Army Medical Research Laboratory, Fort Knox, Kentucky* 


I. INTRODUCTION 


the retinal size of a familiar object 
as a cue to distance. A distinction is 
sometimes made between apparent abso- 
lute distance and apparent relative dis- 
tance. Apparent absolute distance is the 
apparent distance of an object from the 
observer. Apparent relative distance is 
the apparent depth between objects 
without reference to the position of the 
observer. The characteristic which dis- 
criminates between these two types of 
apparent distance is the reference point 
or point of zero distance with respect to 
which the perception occurs. If this ref- 
erence point involves the apparent posi- 
tion of the observer, an absolute-distance 
perception has occurred. If this reference 
point is independent of the apparent po- 
sition of the observer and only involves 
objects external to the observer, relative- 
distance perception is involved. 
is considered to 
f absolute distance 
objects. For a con- 
1 (angular) size of 


T present study is concerned with 


One of the factors which 
be a cue to the perception о 
is the retinal size of familiar 
stant physical size, the retina. 1 | Г 
an object decreases with increasing distance rom 
the observer, Familiarity with a certain physical 
size of an object at various distances might re- 
sult in a distance being perceptually ascribed 
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to each retinal size of the familiar object. When 
the observer is presented with a familiar visual 
object of a particular retinal extent, the per- 
ception might be that the object is at a particu- 
lar distance, with this perceived distance being 
related to the retinal size of the object (5, 6). 
This will be called the size-distance hypothesis. 
According to this viewpoint, a familiar visual 
stimulus of a particular retinal size will be 
seen at the distance position which a normal- 
sized object of the same familiar category would 
have to occupy in order to have the particular 
retinal size. Within limits, if the retinal size 
doubles, the perceived distance should halve; 
if the retinal size is halved, the perceived dis- 
tance should double, etc. It is a problem of this 
study to investigate whether the retinal sub- 
tense of a familiar object can act as a determiner 
of the apparent absolute distance of that object 
from the observer. 


If two similar familiar objects of differ- 
ent retinal size are presented simultane- 
ously, the one with the smaller retinal 
subtense may appear to be more distant 
than the other. This might occur even 
though neither object is perceived as be- 
ing at any particular absolute distance. 
In this case, the perceived-distance dif- 
ference of the two objects would involve 
a relative- not an absolute-distance per- 
ception. If the two objects were pre- 
sented successively, the same results 
might occur and the same interpretation 
be applied to these results. In the case of 
successive presentation, the memory of 
the retinal extent of the first object could 
be compared with that of the second ob- 
ject. With either the simultaneous or 
successive presentation of objects, rela- 
tive retinal size might x і - 
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ceived relative distance. In investigating 
whether the retinal subtense of a familiar 
object is an effective cue to its absolute 
distance, the relative-size cue must not 
be available between objects presented 
in the test situation. This indicates two 
requirements for determining the rela- 
tionship between familiar retinal sub- 
tense and perceived absolute distance: 
(a) a test situation must be used in which 
only one object is presented visually to 
the subject; and (D) the different retinal 
subtenses of the familiar object must be 
presented to different subjects. 

The problem of determining whether 
an absolute- or a relative-distance per- 
ception (or both) are involved in a par- 
ticular distance perception is difficult. 
Pertinent to this problem is a considera- 
tion of the response by which the dis- 
tance perception is measured. The first 
requirement, stated at the close of the 
preceding paragraph, for finding the re- 
lation between the retinal subtense of a 
familiar object and perceived absolute 
distance suggests that a  nonvisual 
method of measuring perceived absolute 
distance be used. In the present study a 
motor performance was employed. The 
subject was asked to throw darts to the 
distance of an object without being able 
to see the results of the throw. This task 
would ordinarily require that the sub- 
ject judge the distance of the object from 
himself. However, even the results of this 
task must be interpreted with caution. 
If, for example, an object appears in- 
definitely located in distance, a subject, 
when given the task of throwing to the 
distance of this object, may make some 
throwing response. If, subsequently, an 
object is presented which appears to be 
less distant than the first object, the sub- 
ject may throw to a lesser distance. This 
may occur even though both the first and 
the second object are perceptually in- 


definitely located in their distance from 
the subject. In this case, the difference 
between the throwing to the two suc- 
cessively presented objects would indi- 
cate the perceptual depth between them. 
The change in the throwing scores could 
be considered a measure of relative and 
not of absolute distance perception. Con- 
sequently, for any subject, only the 
throwing to the object which is first pre- 
sented may be considered in measuring 
the perceived absolute distance of that 
object. This is in agreement with the 
second requirement stated previously. 


The experimental evidence for the occurrence 
of relative- and absolute-distance perception as 
a function of retinal size has been reviewed by 
Ittelson (3). Several studies have seemed to in- 
dicate that the retinal size of a familiar object 
is a cue to its absolute distance. Hastorf (1) 
had subjects adjust the size of a monocularly 
observed object until it appeared to be equi- 
distant with a post in a binocularly observed 
field. He found that the adjusted size of the 
monocular object differed directly as a function 
of its assumed physical size, in order for it 10 
appear at the same distance position as the post- 
He also found that the apparent distance posi- 
tion of the monocular object with respect to 
the post changed as its assumed size was changed- 
Ittelson апа Ames (4) were interested in de- 
termining the effect of apparent distance upon 
accommodation and convergence, The apparent 
distance of a playing card was measured by ad- 
Justing a movable post to the perceived dis 
tance of the card. The retinal size of the play- 
ing card was varied by the experimenters. It 
was found that, for the same physical distance 
of the playing card, the adjusted position 0 
the post varied inversely with the retinal sub- 
tense of the playing card, Ittelson (3) had sub- 
Jects adjust a movable package of cigarettes of 
а checkerboard design to the apparent distance 
position previously occupied by an experimenta 
object. In one experiment two sizes of playing 
cards, ink blots, and diamond-shaped figures 
were used as experimental objects. The assumed 
size and physical distance of the experiment 
object was kept constant while its retinal 5126 
жаз varied. In a second experiment using play- 
ing cards, a typewritten letter, and a match 
box as experimental objects, either or both 
assumed size and retinal size were varied with 
Physical distance held constant. In both €x- 
periments, when familiar experimental objects 
were employed, the movable object was adjusted, 
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in general, to a distance position which a normal- 
sized object of the same kind as the experi- 
mental object would have to occupy, in order 
to have the same retinal subtense as the experi- 
mental object. 

The results of these three studies seem to in- 
dicate that the retinal size of a familiar object 
сап determine its apparent absolute distance. 
But, in none of these studies has the possibility 
been eliminated that the perception which was 
measured was a perception of rclative, not ab- 
solute, distance. The method of measuring per- 
ceived absolute distance involved an apparent 
distance judgment between an experimental: ob- 
ject and another object (a comparison object) 
with both objects visually presented. In the ex- 
perimental field the only cue to distance was 
the retinal size of the experimental object. The 
comparison object was presented in а generally 
well-structured comparison field. Regardless of 
whether the experimental and comparison ob- 
jects were presented simultaneously or successive- 
ly, this method permitted relative-size judgments 
to be used in making the distance judgment. 

Consider the case in which a familiar object 
Was present in both the experimental and com- 
parison field. For example, in one part of the 
study by Ittelson (3), a package of cigarettes in 
the comparison field was adjusted to the ap- 
parent distance position previously occupied vi 
а 114-size playing card in the experimental fied. 
The resulting distance adjustment of the package 
of cigarettes was to a distance which was ap- 
proximately two-thirds of the physical distance 
of the playing card (3, p. 63). This result саз 
be explained by assuming that the subject ad- 
justed the distance of the cigarettes until à 
ratio of the retinal size of thc cigarette package 
and playing card was reached which was pen 
normally experienced by the subject when Eur 
dinary objects of these types were at the sam 
distance from himself. If this explanation 15 
correct, the perception of absolute distance was 
not involved in the adjustment. = 

Consider the case in which a ai no 
ject was used in the comparison V "uh 
familiar object in the experimental iela. ) 
example, in the study by Ittelson and aes 
the comparison object was а movable роз ке 
the experimental object was а playing e F Ae 
study by Hastorf (1) also is an examp! p us 
Case. The assumption of the QUEE. wp 
experimental object was, for р, = P E 
Pong ball made the experimental objec died 
miliar object. To the extent that the compas 
field was sufficiently well-structured to P йе 
information as to the physical size of t EI 
a relative size judgment between ea ee sed 
the familiar experimental object coul к ре 
in adjusting the post and experimental 05) 
to apparent equidistance. 


These three studies do not meet the first re- 
quirement given previously for testing the rela- 
tion between size and perceived absolute dis- 
tance. The second requirement was also not met, 
in that the same subjects were used in more 
than one test situation. Subject comparisons of 
the retinal sizes of experimental objects between 
subsequent test situations may be expected to 
result in the perception of relative distance. 
"That this will occur even when the experimental 
objects are not familiar is suggested by the re- 
sults which Ittelson obtained when ink blots 
and diamond-shaped figures were used as ex- 
perimental objects. The apparent distance of the 
experimental object, as determined by the ap- 
parent equidistance adjustment of the compari- 
son object, was less for the larger than for the 
smaller experimental object of similar shape (3, 
p. 63). The adjustment of the comparison object 
in the initial situation would be to an indifferent 
distance. The adjustments of the comparison 
object in subsequent situations, involving an 
experimental object of the same shape but dif- 
ferent size, would be inversely related to the size 
change of the experimental object. 


II. APPARATUS AND PROCEDURE 


In the present study, the throwing of 
darts was used to measure the perceived 
distance of an object in situations in 
which, ideally, only the retinal size of a 
familiar object was present as a possible 
cue to distance. From the experimental 
result of thrown distance, perceived dis- 
tance was to be inferred. For this reason, 
it was considered desirable to specify the 
magnitude of the throwing response as a 
function of the perceived distance. This 
was experimentally determined by using 
situations in which a variety of cues to 
distance were available. These situations 
are called “full-cue situations." The situ- 
ations in which it was attempted to elimi- 
nate all cues to depth but that of retinal 
subtense are called "reduced-cue situa- 
tions." It is assumed that in the full-cue 
situations the distance of the object from 
the observer was correctly perceived. Any 
difference in the full-cue situations be- 
tween the physical and the thrown dis- 
tance was attributed to a lack of one-to- 
one correspondence between visually per- 


LAAT ADAPTATION 


Fic. 1. Schematic representation of the throw- 
ing and visual alley for the full-cue situations. 
ceived distance and thrown distance, and 
not to any inaccuracy in the perceived 
distance. The relation between thrown 
and visually perceived distance, which 
was experimentally determined from the 
full-cue situations, was used to convert 
thrown distance to perceived distance in 
the reduced-cue situations. 


A. The Reduced-Cue Situations 

1. Apparatus 

The experimental situations of this 
study were presented in a visual alley 
separated from an adjacent throwing 
alley by a partition of dark cloth. Figure 
1, which illustrates the full-cue situa- 
tions, can also be used to discuss the re- 
duced-cue situations. Both the visual and 
throwing alley were 4 feet wide and 4o 
feet long. A small rectangular aperture 
(1 inch by 5 inches), located 15 inches 
above the surface of the visual alley and 
at the same height as the surface of the 
throwing alley, allowed the subject to 
see into the visual alley. The surfaces of 
the visual alley and the surface contain- 
ing the viewing aperture were dark. The 
ladder pattern shown on the floor of the 
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visual alley in Fig. 1 was present only in 
the full-cue situations. 

The familiar object used in the re- 
duced-cue situations was produced by 
seven-of-spades photographic transparent 
positives. One transparency was of a 
normalsized card (2-14 inches by 3/6 
inches) and the other was of a double- 
sized card (4-14 inches by 7 inches). Each 
of the two transparencies was fitted on 
the face of a light box and transillumi- 
nated with white light. No light from the 
box was visible except that passing 
through the transparency. When viewed 
in an otherwise dark room, a lighted 
transparency appeared to be an illumi- 
nated playing card and will be referred 
to as a playing card throughout this 
study. When a playing card was pre 
sented in the visual alley, the bottom of 
the card was at the height of the floor 
of the throwing alley. 

Only one card was present in any one 
situation. Four reduced-cue situations 
were used. These will be designated by 
letter as follows: 


А-А double-sized card at 10 fect. 
B—A normalized card at 10 feet. 
С—А double-sized card at 20 feet. 
D—A normal-sized card at 20 feet. 


Considerable care was taken with the 
reduced-cue situations so that no part 
of the visual alley or any object except 
the single playing card was visible to the 
subject when he looked through the 
viewing aperture. Black baffles were 
placed in the visual alley so that light 
from the playing card which fell on the 
floor and walls of the alley would not be 
visible. As an additional precaution, the 
subject was light-adapted before looking 
through the viewing aperture. The light- 
adapting surface (see Fig. 1) had a 
brightness of 11 foot-lamberts. The 


brightness of each playing card was 1-1 
foot-lamberts. 
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The cloth partition between the visual 
and throwing alley prevented the sub- 
jects from seeing where the thrown darts 
landed. This was necessary since, if the 
thrown darts had landed in the field of 
view, relative-distance judgments be- 
tween the card and dart could have oc- 
curred. The darts used in the throwing 
were 6 inches long and weighed approxi- 
mately 11 grams. The distance to which 
the darts were thrown in the throwing 
alley was measured to the nearest centi- 
meter. 


2. Procedure 


Eighty men who were right-handed and had 
at least 20/20 corrected vision in their right eyes 
were used as subjects. Prior to being subjects, 
these men were not informed of the purpose of 
the experiment. In the reduced-cue situations 
the viewing was monocular with the left eye of 
the subject covered. Before looking into the 
visual alley, the subject was given 20 practice 
throws to accustom himself to the throwing task. 
The throwing alley was lighted for this рр = 
Following, this, all lights except that рш 
adaptation surface and playing card were turned 
off. A shutter covered the viewing aperture. A 
mark on this shutter allowed the subject to ori- 
ent his head so that his eye would be in the 
correct viewing position when the shutter was 
raised. The subject viewed the ndaptaHon $i. 
face for approximately five seconds, after w hich 
he oriented his head with respect to the mute 
mark and the adaptation surface was turned ed 
The shutter was raised and the subject кес 
at the playing card. By questioning, jt was = 
certained that the subject saw the аш з 
seven-of-spades playing card, The su nject kie 
reminded of his task which was to throw : A 
so that it landed at the distance of the imd E 
card. He was also reminded that the bigs 
the bottom of the card was the same as U 0. 
the floor of the throwing alley. The Er im 
handed a dart, which he threw down the EA à 
ing alley. The shutter was then closed, the adap 


i i and the subject again 
tation. light turned on, anc on sa ae 


light-adapted. This ps alley, and throw- 
tii ki i he viewing à , х 
on, looking into t S th та successful 


ing a dart was repeated : 
(scorable) throws were obtaine 
lar subject in the particular situ: dud 
ally, a dart either did not land on oF 5 sae 
the floor of the throwing alley. Pues a 
was not considered to be successful and was 


Scored. 


d for the particu- 
ation. Occasion- 


Each of the 80 subjects was presented with 
each of the four situations, A, B, C, and D. А 
four-minute interval occurred between the end 
of the presentation of one situation and the be- 
ginning of another. The order in which the 
four situations were presented was varied be- 
tween subjects. Since 24 orders were possible, 
these 24 orders were used three times (totaling 


72 subjects), with four orders and their inverse 


selected again from the 24 possible orders, such 
that each situation (A, B, C, or D) was pre- 
sented first equally often. Thus, of the 80 sub- 
jects, 20 were first presented. with the reduced- 
cue situation. А, 20 other subjects were first 
presented. with B, 20 others with C, and 20 


others with D. 

Following the 10 scorable throws to the ap- 
parent distance of the card in a reduced-cue 
situation, the subject was asked to estimate the 
distance to the card in feet. 


B. The Full-Cue Situations 


1. Apparatus 

For the full-cue situations, the fluores- 
cent lights above the visual alley were 
turned on. There were no lights used in 
the throwing alley and the results of the 
dart throwing were not visible to the 
subject. The walls and floor of the visual 
alley were visible when the subject 
looked through the viewing aperture. A 
38-inch-wide strip of brown paper was 
placed on the floor of the visual alley. 
Fastened to this strip of paper was a 
ladder pattern formed from a 1-іпсһ- 
wide white tape. The sides of this pat- 
tern were separated by 28 inches, with 
the horizontal extensions occurring at 
18-inch intervals. This pattern, part of 
which can be seen in Fig. 1, extended 
from the subject to a distance of 26.5 
feet. A normal-sized seven-of-spades play- 
ing card (not a transparency) was 
mounted on a stand and placed at either 
5, 10, 15, 20, OF 25 feet from the subject. 
The normal-sized card located at each of 
these five distances constituted five full- 
cue situations. The bottom of the playing 
card was always at the height of the 
floor of the throwing alley. The subject 
binocularly observed the visual alley, in- 
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IN FEET. 


20 


THROWN AND VERBALLY ESTIMATED DISTANCE 
5 
T— 


pop — 


LL 


5 10 15 20 25 
PHYSICAL DISTANCE OF TARGET IN FEET 
Fic. 2. Relation between physical and behav- 
ioral distance from all presentations of the 
full-cue situations, 


cluding the playing card, through the 
viewing aperture. The light adaptation 
surface was not used in the full-cue situ- 
ations. 


2. Procedure 


The same 80 subjects were used in the full- 
cue situations as had been used previously in 
the reduced-cue situations. The full-cue situa- 
tions were always presented after the reduced- 
cue situations so that the subjects would be 
naive when they were first presented with the 
reduced-cue situations. Each subject completed 
10 scorable dart throws to the apparent distance 
of the card in each of the full-cue situations. 
As in the reduced-cue situations, the task of 
the subject was carefully explained and he was 
reminded that the bottom of the card was at the 
height of the floor of the throwing alley. The 
shutter was closed over the viewing aperture 
between throws. A four-minute interval occurred 
between the end of one full-cue situation and 
the beginning of another. All five full-cue situa- 
tions were presented to each subject, but the 
order in which the five full-cue situations were 
presented was different for each subject, Forty 
orders and their inverse were used. There were 
16, 16, 17, 15, and 16 different subjects who 
were first presented with the target in the full- 
cue situations at 5, 10, 15, 20, and 25 feet re- 
spectively. For each subject, the time interval 
between the end of the last reduced-cue situa- 
tion and the beginning of the first full-cue 
situation was 7 minutes, The experimental order 
for each subject throughout the entire experi- 


ment was specified by randomly pairing the 
order in which he was to sec the reduced-cue 
situations with one of the 80 orders of pre- 
senting the full-cue situations. 

Following 10 scorable throws to the apparent 
distance of the card in any full-cue situation, 
the subject was asked to estimate the distance 
to the card in feet. 


ПІ. RESULTS 
A. The Full-Cue Situations 


The results from the full cue situa- 
tions are presented first, since these re- 
sults produced the calibration equations 
used in the analysis of the reduced-cue 
situations. The average throwing scores 
and verbal estimates from all presenta- 
tions of the full-cue situations as a func- 
tion of the physical distance of the play- 
ing card are shown in Fig. 2. The dotted 
line represents the results which would 
have been obtained if the throwing and 
verbal scores were the same as the physi- 
cal distances of the playing card. Each 
thrown distance represented by a point 
on the ordinate of Fig. 2 is a mean of 
80 scores, one from each subject, where 
each score is an average of 10 dart 
throws. Each verbal estimate represented 
by a point on the ordinate is a mean of 
80 verbal estimates with one estimate 
from each of the 8o subjects. The graph 
involving the verbal estimates is closer 
to the dotted line than is the graph in- 
volving the thrown distance scores. How- 
ever, the average of the between-subject 
standard deviations of the verbal esti- 
mates (12.9 feet) is larger than that from 
the throwing scores (2.1 feet). The aver- 
age standard deviation of 12.9 feet is 
inflated by the verbal responses of оле 
subject. When the verbal data of this 
subject are excluded, this standard devi- 
ation reduces to 4.2 feet. In order to 
estimate the precision with which the 
throwing method differentiated between 
target positions, standard errors of dif- 
ferences were computed for the differ- 
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ences between throwing scores when the 
targets were physically separated by five- 
foot intervals. Averaging the four result- 
ing standard errors gave a value of .2 
feet. This suggests that, in the full-cue 
Situations, a distance difference of .3 feet 
could have been statistically differenti- 
ated by the dart-throwing at the .05 level 
of confidence (using the single-tailed ¢ 
test). 


B. The Reduced-Cue Situations 


As was mentioned previously, the re- 
lation between thrown and visually per- 
ceived distance was determined from the 
results of the full-cue situations and was 
used to convert thrown distance to per- 
ceived distance in the reduced-cue situa- 
tions. This was accomplished as follows: 
By the method of least squares a linear 
relation was calculated, for each subject, 
between the dart-throwing scores and the 
physical distance of the card in the full- 
cue situations. It was assumed that physi- 
cal and perceived distance was equiva- 
lent in the full-cue situations. This re- 
sulted in a linear equation for each 
subject which expressed throwing dis- 
tance as a function of perceived distance. 
This function is called the calibration 
equation for a subject. Each throwing 
Score (a mean of 10 throws) from each 
subject in each reduced-cue situation was 
transformed by the calibration equation 
to а “perceived-distance score.” Actually, 
the perceived-distance scores can be con- 
sidered to represent а perceived absolute 
distance only if the perception of abso- 
lute distance occurred in the reduced-cue 
situations. If the perception of an abso- 
lute distance was absent in the reduced- 
Cue situations, the throwing would be to 
an indifferent distance. However, 1n 
Order to investigate whether the percep- 
tion of absolute distance varied with 
changes in the retinal size of a familiar 


object, this transformation is logically to 
be preferred. 


1. First-Presented Reduced-Cue Situa- 
tions 


As discussed in the introduction, in 
determining whether the retinal sub- 
tense of a familiar object is a cue to ab- 
solute distance, only one object should 
be visually present in the test situation, 
with similar objects of different retinal 
subtense being presented to different 
subjects. The results from each group of 
20 subjects, who saw a particular re- 
duced-cue situation first, meet both of 
these requirements. "Therefore, the re- 
sults from these first presentations of the 
reduced-cue situations were analyzed 
separately from the remaining results 
from the reduced-cue situations. 

According to the size-distance hypothe- 
sis, the playing card in the reduced-cue 
situations A, B, C, and D should appear 
at 5, 10, 10, and 20 feet respectively. Let 
Pis Py; Po and Pp, represent the per- 
ceived distance of the playing card in 
the first presentations of the reduced-cue 
situations A, B, C, and D. The sub- 
script "1" refers to first presentations. 
The predictions from the size-distance 
hypothesis are that P4, — 5 feet, Py, — 
10 feet, Pc, = 10 feet, and Pp, = зо feet. 
A more general but less stringent 
hypothesis relating retinal size and 
perceived absolute distance is that 
p, = Py = Po < Py, If the results 
from this study do not support the latter 
hypothesis, they also do not support the 
former hypothesis. The analysis of the 
results will be mainly with respect to the 
latter, more general, hypothesis. 

Table 1 gives the average results 
which were obtained from the first pres- 
entations of the reduced-cue situations. 
The entries in the column labeled 
“Thrown-Distance Score" are based upon 
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TABLE 1 
RESULTS IN FEET FROM THE First PRESENTATIONS OF THE REDUCED-CUE SITUATIONS 
(N=20) 
Thrown-Dis- | Perceived-Dis- | Verbal | . . | и" 
Reduced-Cue | tance Score | tance Score Report | Size-Distance Physical 
Situation \ Prediction | Distance 
| Mean SD | Mean SD | Mean SD = 
A 16.4 4.4 22.4 7.1 | 22.4 18.0 | 5 | 10 
B 14.8 4.6 EE © | X43, Sax 10 | 10 
Cc 12:4  $,0 25.9 10.1 | 19.0 12.7 | 10 20 
D 19.2 5.1 290.9 9.9 | 20.2 13.9 20 | 20 


20 scores, with one score from each of 
20 subjects, where each score is a mean 
of то throws of the darts. The entries in 


the column labeled "Perceived-Distance 


OF THE 


Score" were obtained by applying the 
appropriate calibration equations to the 
thrown-distance scores. The column lab- 
eled “Size-Distance Prediction" gives the 
distances at which the playing cards 
should have been seen according to the 
size-distance hypothesis. The column 
labeled “Physical Distance” gives the ac- 
tual distances at which the playing cards 
were placed. 

From Table 1 it appears that neither 
the thrown distances, the perceived 
distances, nor the verbally reported 
distances are in agreement with all the 
relationships involved in P}, < Py, = 
Po, < Pp,- The statistical significance of 
the differences between the perceived- 
distance scores of Table 1 was deter- 
mined by a simple analysis of variance. 


TABL 


SIGNIFICANCE OF THE Dir 
THE First PRE 


ENCES BETWEEN PERCEIV 
ATIONS OF THE REDUCED-CUE Situ 


- Mean | " 
Source of Variance | df | Saue | F 
Between Situations | 8 | 248,968 | 3.22 
Within Cells | 76 | 77,346 
Total | 79 | | 


The results of this analysis are shown in 
Table 2. The obtained F is significant 
between the .o5 and .o1 level of confi- 
dence. 

The significance of the differences be- 
tween pairs of perceived distances Pay 
Prp Poy, and Pp, were tested using the 
within-cells variance from the analysis 
of variance in calculating the error term. 
The results of these tests for single-tailed 
t probabilities are shown in Table 3: 
The direction of the subtractions in the 
column labeled “Test” was derived from 


E 3 


ED-DISTANCE SCORES FROM 


ATIONS __ 

| Differ zu m A 
Test | alitative Di ы ifferen я 

ез! | Qualitative Differences | on үне | i | arvo 

. Pg, = Paz | Angle | 5 Е 

" Pp,—Pe,>0 | Angle | e: к | 1 ever! 
з. Pe, - P» 20 Distance 18 ч = 
4. Pc, —Pa,>e | Angle and Distance Жык. | ds | = 
5. Pp, - Pn, 20 Angle and Distance XE 2 2 
6. Pp —Pa,>0 | Angle and Distance r% PE 2^8 
7. (Pp, — P4,) - (Pe, = Рв) >o | Angle Corrected for Distance 37 res | vel 


а Single-tailed probability of occurrence. 


à 


Û 
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the predictions of the size-distance hy- 
pothesis. The word “Reversal” in the 
column labeled “р” means that the ob- 
tained difference was opposite in direc- 
tion to that predicted from this hypothe- 
sis. The column labeled “Qualitative 
Differences" in Table 3 indicates 
whether the playing cards in the two 
reduced-cue situations differed from each 
other in either or both angular (retinal) 
size and distance from the subject. It 
will be observed from test 1 and test 2 
of Table that, by itself, a decrease 1n 
the visual angle of the playing card did 
not produce a statistically significant in- 
crease in perceived distance. But, in 2 
ош of $ cases, a decrease in visual angle 
accompanied by an increase in physical 
distance did produce a significant in- 
crease in perceived distance (see tests 4, 
5, and 6). It is reasonable to expect that, 
in spite of the attempt to eliminate all 
distance cues except that of visual angle, 
some residual effect of other distance 
cues remained in the reduced-cue situa- 
tions. This effect would be reflected in 
the results of test g in which the visual 
angle of the playing card was held con- 
stant but its physical distance was 
changed. As shown by the results of = 
3, when the physical distance of the play- 


i ; ived- 
ing card was increased the perce! 


ч $ by a 
distance score increased, but not by 


statistically significant amount. p 
concluding that the significant. di A 
ences found in tests 5 and 6 can be 
tributed to the change in visual angle о 

the playing card, it must be ny 
strated that these results cannot gar 
tributed to the possible presence of or 
distance cues. For this reason, the au er 
ence found in test 3 Was subtracted fr ji 
the differences found in tests 5 pom 
Actually, the subtraction of the rs з 
ence found in test $ from that RS 

test $ is unnecessary, since the resulting 


difference is the same as that from test 2. 
The subtraction of test 3 from test 6 is 
shown as test 7 in Table 3. It will be seen 
that the difference in test 7 was not sig- 
nificant at the .05 level of confidence. It 
can be concluded that the perceived- 
distance scores of Table 2 provide no 
evidence that the retinal subtense of the 
playing card was used as a cue to deter- 
mine the perception of the absolute 
distance of the card. 

An analysis of variance also was per- 
formed on the throwing scores of Table 
1. The F value of this analysis was sig- 
nificant beyond the .o5 level of confi- 
dence (F = 3.07). The significance of the 
differences between pairs of these throw- 
ing scores was tested by the procedure 
indicated in Table 3. The single-tailed 
probabilities resulting from the ¢ tests 
were similar to those obtained from the 
analysis of the perceived-distance scores, 
with the only difference of any impor- 
tance being that test 3 was significant 
beyond the .05 level of confidence 
(t = 1.93) when the throwing rather than 
the perceived-distance scores were used. 
Neither the throwing scores nor the per- 
ceived-distance scores offer support for 
the hypothesis that the retinal size of the 
familiar object was used as a cue to its 
absolute distance. 


ə. All the Reduced-Cue Situations 


It will be recalled that the reduced-cue 
situations A, B, C, and D were each pre- 
sented successively to each of the sub- 
jects. So far, the analysis of the data has 
been only with respect to the results from 
the first presentations of the reduced-cue 
situations. The average results from all 
presentations of each of the reduced-cue 
situations are given in Table 4. Each 
mean and standard deviation of Table 4 
is based upon 80 scores, with one score 


from each of the same 8o subjects, The 


у. Reseal 
pavi af HNG COLLE 


Bure? Edni. 
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TABLE 4 
RESULTS IN FEET FROM ALL PRESENTATIONS OF THE REDUCED-CUE SITUATIONS 
(N=80) 
Thrown-Dis- | Perceived-Dis- Verbal | . " | — 
Reduced-Cue | tance Score tance Score Report | Size-Distance | Physica 
Situation Prediction | Distance 
Mean SD | Mean SD | Mean SD | 
A 2.5 3.8 16.5 8.6 12.4 8.0 5 10 
в 14.9 1.6 21.34 7:9 16.3 11.9 10 10 
G ї$.б а 1,7 22.8 09.5 21.1 13.0 10 20 
D X9o.0 Eg 28.4 8.0 26.1 19.7 20 20 
calibration equations were used to con- TABLE 5 


vert throwing scores to perceptual scores. 
An analysis of variance was performed 
on these perceived-distance scores with 
the results shown in Table 5. The F ob- 
tained for the main effect (situations A, 
В, С, and D) was significant beyond the 
001 level of confidence. The results of 
the one-tailed ¢ test of the significance of 
the differences between pairs of the per- 
ceived-distance scores of Table 5 are 
given in Table 6. It will be observed 
that all the differences in Table 6 are 
significant beyond at least the .025 level 
of confidence. The perceived-distance 
scores varied inversely with the retinal 
size of the playing card between succes- 
sive presentations of the reduced-cue sit- 
uations. As indicated by tests 1, 2, and 7 
of Table 6, these results cannot be ac- 
counted for by the effect of distance cues 
other than changes in retinal size. 


A parallel analysis was completed for 


SUMMARY OF ANALYSIS OF VARIANCE OF THE PER- 
CEIVED-DISTANCE SCORES FROM ALL 
PRESENTATIONS OF THE REDUCED- 

CUE SITUATIONS 


"Tw | P Mean F 
Source of Variance | df Square 
Between Situations 3 1,775,177, 81-47 
Between Subjects 79 215,149| 9.97 
Situations X Subjects | 237 21,789 
Total 319 


the throwing scores of Table 4. The € 
sulting F and t values were very similar 
to those which occurred with the per 
ceived-distance scores in Tables 5 and 6. 
Both the throwing scores and the per 
ceived-distance scores support the hy- 
pothesis that the visual angle of the play- 
ing card significantly modified the per 
ceived-distance position of the card fro? 
one reduced-cue situation to anotht? 
when the same subjects were used in the 
several situations. 


TABLE 6 


SIGNIFICANCE OF THE DIFFERENCES BETWEEN PERCEIVED-DISTANCE SCORES FROM 
ALL PRESENTATIONS OF THE REDUCED-CUE SITUATIONS 


Test 


А + H i s 

Qualitative Differences Difference t EAT 
ыш == 

т. Pa—Pa >o Angle oie 

pam 2 ‘Angle 4.8 6.24 2 or 

з. Рс—Рв>о Distance E dp 6028 

4. Рс-Рл>о Angle and Distance EAE $2 < .01 

5. Ро—-Рв>о Angle and Distance B ee us 

6. Pp- P420 Angle and Distance (lee oe <.о1 

т. (Ръ=Рл) –(Ре-Рв) >о | Angle Corrected for Distance 154 ie bi 


* Single-tailed probability of occurrence, 
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The analysis of the results from the 
first-presented reduced-cue situations 
(Tables 2 and 3) gave no indication that 
absolute-distance perception had oc- 
curred as a function of the retinal size 
of the playing card. The analysis did 
suggest, however, that some perception 
of absolute distance had occurred as a 
function of distance cues other than 
those of retinal size. These cues would 
operate to make the physical and ap- 
parent distances of the playing cards 
coincide. The degree to which the physi- 
cal and apparent distances are not In 
agreement indicates the extent to which 
these residual cues of absolute physical 
distance were unimportant in producing 
perceptions of absolute distance in the 
reduced.cue situations. This can be eval- 
uated by comparing the perceived- 
distance scores of Table 1 with the actual 
physical distances of the cards. It will be 
observed from Table 1 that, when the 
cards were at 10 feet, the perceived- 
distance scores were greater than 20 feet. 
That the perceived-distance scores are 
not proportional to the physical dis- 
tances in the first presentations of the 
reduced-cue situations is also suggested 
by Table 1. Computing the ratio of the 
largest to the smallest perceived-distance 
score in Table 1 gives à value of 1.4. г» 
corresponding ratio of the physica 
distances is 2.0. It may be concluded that 
the residual cues to the physical distance 
of the playing cards were not very po 
tive in producing perceived absolu 
distance in the reduced-cue situations 
The perception of absolute distance in 
the reduced-cue situations will be re- 
garded as having been largely indie 
Consequently, it will be considered | s 
the subject threw to what was essentially 
ап indifferent distance when first pre- 
sented with a reduced-cue situation. This 
throwing distance was not indifferent, 


however, in that a throwing force was 
used which was definite, and which 
probably was peculiar to the particular 
subject. Later, the same subject, upon 
being presented with a second reduced- 
cue situation, modified the throwing 
force depending upon whether the card 
in the second presentation appeared to 
be more or less distant than that in the 
first presentation. Consider the case in 
which a subject was presented with the 
reduced-cue situations in the order A, B, 
C, D,, where the subscripts refer to the 
order of presentation. Upon being pre- 
sented with A,, the darts would be 
thrown to some indifferent distance. 
When the same subject was presented 
with B,, he behaved as though he re- 
membered both the retinal subtense of 
A, in relation to that of B, and the 
force of the throwing he had used in A,. 
Since the card.in B, was retinally smaller 
than that in A, the card in B, was 
judged to be more distant than the pre- 
viously indefinitely perceived distance of 
the card in A,, and the force of the 
throwing was increased. The throwing 
on C, and D, for the same subject would 
be determined in a similar manner. For 
the order of presentation A, В, C; D,, 
the relative magnitude of the throwing 
would be Ta < Ts, = To Гр 
Since the perception of an absolute 
distance as a function of retinal sub- 
tense was not demonstrated by the analy- 
sis of the results from the first-presented 
reduced-cue situations, the perception of 
distance which occurred between succes- 
sively presen ted reduced-cue situations as 
a function of retinal subtense is con- 
sidered to be relative-distance percep- 
tion. The results indicate that relative- 
distance perception, as some function of 
relative retinal subtense, occurred be- 


tween the successively presented reduced- 


cue situations. 
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If relative-distance perception Oc- 
curred as a function of the relative 
retinal subtense of the playing cards, 
it would be expected that the order of 
magnitude of the throwing scores (or 
perceptual scores) in the successively pre- 
sented reduced-cue situations would be 
independent of the order of presentation. 
This can be demonstrated by considering 
the case in which a subject was succes- 
sively presented with the reduced-cue sit- 
uations in the order D, С, B, Ay. After 
making a perceptually indifferent throw- 
ing score on D,, the subject, when pre- 
sented with C,, should produce a throw- 
ing score which was less than that which 
resulted from D,. This would be a con- 
sequence of the retinal subtense of the 
card in C, being greater than that of the 
card in D,. Similarly, A, should have the 
smallest throwing score of the four situa- 
tions. The relative magnitude of the 
throwing scores would be Ta, < Ty, = 
Те, < Гру As discussed above, this is 
the same order of relative magnitude of 
throwing scores which would be expected 
from the order of presentation A, B, C, 
D,. Continuing this, it will be seen that 
the order of magnitude of the throwing 
(or perceptual scores) would be inde- 
pendent of the order of presentation of 
the reduced-cue situations. It is this ex- 
pectation which, in the comparative ab- 
sence of absolute-distance perception in 
the first-presented situations, permits the 
analysis of the results from all the pres- 
entations of the reduced-cue situations 
to be interpreted. 


3. The Independence of Absolute- and 
Relative-Distance Perception 


In the introduction, it was suggested 
that the characteristic which discrimi- 
nates between apparent absolute and 
apparent relative distance is the refer- 
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ence point with respect to which the 
perception occurs. This reference point 
is some function of the position of the 
observer for absolute-distance percep- 
tion, but is independent of the position 
of the observer for relative-distance per- 
ception. It has been pointed out that ex- 
perimental results which have been at 
tributed to an absolute-distance percep- 
tion might be explained by a relative-dis- 
tance perception. Conversely, the differ- 
ence between two absolute-distance per 
ceptions might be considered, mis- 
takenly, to be a perception of relative 
distance, even though the perceptual 
reference point in this case was not in- 
dependent of the position of the ob- 
server. The results from the reduced- 
cue situations do not support the hypoth- 
esis that perceived absolute distance 
occurred in the reduced-cue situations 
as a function of the retinal size of the 
playing card. But the results do indicate 
that relative-distance perception 9e 
curred as a consequence of changes 1n 
retinal size. This suggests that, as à func 
tion of retinal size, relative-distance per 
ceptions might have occurred in the re 
duced-cue situations without a 60" 
comitant occurrence of absolute-distance 
perceptions. This offers the possibility 
that an apparent relative distance be- 
tween two objects can occur independ- 
ently of the perception of the distance 
of either of these objects. Thus, the 
definition of relative-distance perce?” 
tion used in this study may parallel 2 
real perceptual ability. A test of th 
possibility would be a test in which an 
affirmative answer simultaneously WOU 
depend upon the absence of perceive 
absolute distance and the presence 9 
perceived relative distance. 

While the order of magnitude of the 
perceptual (or throwing) scores shou 
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be independent of the sequence of pres- 
entation of the reduced-cue situations, 
this is not the case with the absolute 
magnitude of the perceptual (or throw- 
ing) scores. Consider, for example, the 
sequence of presentation A, B. as com- 
pared with the sequence of presentation 
D, B, and assume, for the moment, that 
all distance cues except that of retinal 
size had been completely eliminated. 
Under the condition that no absolute- 
distance perception occurred in the re- 
duced-cue situations, P4, and Pp, should 
not differ significantly from each other. 
With the sequence A, B. Py, as a con- 
sequence of the smaller retinal size in В, 
than in А, should be greater than Pa. 
With the sequence D, B, Pro should 
be less than Pp,. Therefore, Py, with 
the sequence of presentation A, В» 
should be greater than Pho with the 
sequence D, Bz. This is based upon the 
assumptions that (а) absolute-distance 
perception did not occur with A, and Dı, 
but (b) relative-distance perception ос- 
curred between successively presented 
Situations. Similarly, Pe, with the se- 
quence of presentation A, C, should 
be greater than Pec, with the sequence 
D, C,. Both B, and C, should have re- 
sulted in larger scores when preceded by 
A than by D. 
Actually, as shown ir 
Py, was significantly 
This is attributed to the € 
cues of absolute distance, other than the 
cue of retinal size. To the extent that 
these cues to the physical distance of the 
target in either А and D or B and C were 
present, B, and С should not have re- 
sulted in larger scores when preceded y 
A, than by D,. Larger scores in Вг Rm 
С», as a consequence of being precede’ 
by A, rather than by D» indicate that 
relative retinal size can produce à per 


1 Table 3, test 6, 
larger than Pus 
ffect of residual 


TABLE 7 


AVERAGE PERCEPTUAL SCORES IN FEET FROM 
SITUATIONS В: AND C», DEPENDING UPON 
WHETHER THE PRECEDING SITUATION 
Was А, ок Di 


Results 


Results 
from Cs 


from В» 


| Mean SD | Mean SD 


Prior Situations 


а 25.6 7.6 | 26.6 6.1 
1 17:2 9:9 | 10.2 бет 
Difference 8.4 ا‎ 
Average Difference 75 


ception of the relative distance between 
two objects, which is independent of a 
perception of the absolute distance of 
either. 

In this study there were seven sub- 
jects who had the sequence A, В„; seven 
who had the sequence D, В„; seven who 
had the sequence A, C,; and seven who 
had the sequence D, C,. Table 7 gives 
the average perceptual scores from B, 
and C, depending upon whether the 
preceding situation was А, or D,. The 
average difference of 7.9 feet is the differ- 
ence between the mean of 14 scores on 
В, or Cs, when the preceding situation 
was A, and the mean of 14 scores on B, 
or С, when the preceding situation was 
D, IE is the average amount by which 
the perceptual scores from B, and C, 
were larger as a consequence of being 
preceded by A, rather than by D,. This 
average difference was significant beyond 
the .о1 level of confidence (t = 2.66, df 
= 26) with the single-tailed ¢ test. This 
indicates that the presence of relative- 
distance perception as a function of the 
size of a familiar object can occur in- 


dependently of the occurrence of ab- 


solute-distance perception. 


IV. DISCUSSION 
A. The Full-Cue Situations 


o expect the perception of 


]t is reasonable t 
n the full-cue situa- 


absolute distance to occur i 
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THROWING SCORES FROM SECOND AND ALL 
SUBSEQUENT PRESENTATIONS 


Fic. 3. Relation between the mean throwing 
Scores in feet from the first and all subsequent 
presentations of the full-cue situations. 


tions. It can be demonstrated that, unlike the 
results from the first presentations of the re- 
duced-cue situations, the throwing in the first 
presentations of the full-cue situations was not 
largely indifferent. This can be demonstrated by 
comparing the average throwing scores from the 
first presentations of the full-cue situations with 
the average throwing scores from the remaining 
presentations of the full-cue situations, This is 
shown in Fig. 3 where this relationship is plotted 
for the five distances of the playing card. It will 
be seen that the data reasonably fit a straight 
line. A straight line was fitted to these data by 
the method of least squares. The equation of 
this line is given in Fig. 3. The similarity of this 
equation to the equation y — x indicates that 
the throwing scores in the first and remaining 
presentations of the full-cue situations were 
essentially the same. 


B. Experimental Precision 


This study produced no evidence that any 
relation occurred between the retinal size of 
the familiar object and the perception of abso- 
lute distance. But it cannot be concluded that 
no such relation existed, since this conclusion 
would require the acceptance of the null hy- 
pothesis. This raises the problem of the experi- 
mental precision with which the relation be- 
tween perceived absolute distance and the ret- 
inal size of a familiar object has been investi- 
gated. This experimental precision can be con. 
sidered either from the standard errors involved 
in the study, or from the kind of perceptual 
discriminations which the study was able to 
demonstrate, 
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One possible estimate of experimental pre- 
cision is the error term which was used to test 
the statistical significance of the average score 
difference between first presentations of the re- 
duced-cue situations. This error term of 2.9 
feet would suggest that a perceived difference 
between situations of 4.8 feet (1.645 X 2.9 feet) 
would have to occur before a statistically signifi- 
cant mean difference would be present with the 
one-tailed t test. On rational grounds, however, 
this is not a good estimate of the experimental 
precision with which familiar size was tested as 
a cue to absolute distance. The error term from 
the first-presented reduced-cue situations hen 
vary depending upon the precision with -— 
the perception of absolute distance occurred. 
For example, with no perception of absolute 
distance, the variability between throwing scores 
from the first presentations of the reduced.cue 
situations would be large, since the iron ne 
would not be controlled by a visually perceive 
distance. The problem is to determine an error 
term which reflects the precision of measurement 
which would occur if the perception of absolute 
distance were definite in the reduced-cue situa- 
tions. In the full-cue situations it was assumed 
that the perception of absolute distance was 
accurate. The standard error, calculated from 
the average between-subject variability in E 
full-cue situations, is a possible measure of үе 
precision with which the existence of a 0 
between perceived absolute distance and retina 
size was tested in the reduced-cue situation? 
This was calculated by computing the gander 
deviation of the subjects’ scores in each of pie 
full-cue situations, for each of the four groups 
of 20 subjects who had previously been used in 
the first presentations of the reduced-cue Es 
tions. The resulting average standard deviate 
Was 2.0 feet. Using this average standard devia 
tion, the standard error of the mean difference 
between the results from different groups of a 
subjects is .7 feet. With this standard error, t 
mean difference of 1.1 feet would be significa” 
at the .05 level of confidence, with the en 
tailed ¢ test. This indicates that with two di 
ferent groups of 20 subjects, with each SOS 
presented with a different reduced-cue sitar 
a mean difference between groups of 1.1 feel 
perceived absolute distance would be Sra 
cantly differentiated by the throwing procedu Ж. 
at the .05 level of confidence, As noted prev! 
ously, the error term calculated from the s 
duced-cue situations is probably too large ith 
be considered an estimate of the precision b: а 
which the relation between the retinal size О cs 
familiar object and perceived absolute disan 
was tested. The estimate from the full-cue an nis 
tions, however, is possibly too small for hat 
purpose, since it contains the assumption eu i 
if the perception of absolute distance occur"! 
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in the reduced-cue situations, the variability be- 
tween subjects attributable to this perception 
Would be no greater than the variability which 
occurred in the full-cue situations. To the de- 
gree that additional perceptual variability can 
be tolerated without rejecting the possibility of 
Some gencral relation between retinal size and 
perceived absolute distance, this estimate of ex- 
perimental precision is too small, Therefore, the 
experimental precision with which the occur- 
rence of the relation between the retinal size of 
à familiar object and perceived absolute distance 
Was tested lies somewhere between these two 
Possible estimates, ie, somewhere within the 
range of approximately 1 to 5 fect. y 
It is possible to consider conclusions which 
reject the null hypothesis with respect to any 
Particular relation between retinal size and the 
Perception of absolute distance. One such rela- 
tion is that given by the size-distance hypothesis. 
The perceived distances predicted from the size- 
distance hypothesis for the reduced-cue situations 
are given in Table 1. The perceptual scores of 
Table 1 are logically the values to compare with 
the corresponding predicted distances. The error 
term from the analysis of variance of these per- 
ceived distance scores (2.9 feet) can be used to 
calculate the standard error, and thus the statis- 
tical significance of the differences between the 
Perceptual scores and the corresponding pre- 
dicted scores, If this is done, it is found that 
all of these differences are significant beyond the 
01 level of confidence, by the double-tailed ¢ 
test. This supports the conclusion that the 
magnitudes of the obtained perceptual S 
are not in agreement with the values predicted 
from the size-distance hypothesis i 
Any statistical significance which 
tween! the overeat results of the рейн 
Situations is evidence that the throwing metho 
can measure perceived depth in these иы 
Statistical significance was present between ie 
average results from the successively presente 
reduced-cue situations, indicating the in) 
Of relative-distance perception. The stan si 
error used to test the significance of the оя 
difference between апу two such cents 
8 feet for the perceived-distance scores. 3 
much smaller than the corresponding error m 
Of 2,9 feet from the perceived-distance зое 
With the first-presented reduced-cue "pp 
The experimental precision of the test fo baat 
tive-distance perception was greater тан alo 
for absolute-distance perception. But it wee 
found that the perceived-distance puis тоа 
Situation В ог С varied, depending проп le 7). 
В or C was preceded by А or D peus ce of 
he standard error for the test ОЁ Ps ld 
is comparison was 3.0 feet. This is o ^ of 
Order of magnitude as the standard е 


3 rst-pre- 
Pairs of mean differences from the AGED 


occurred be- 
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sented reduced-cue situations (2.9 feet). This in- 
dicates that, with standard errors of this magni- 
tude, a statistically significant mean difference 
can appear, when the basis for expecting this 
difference is that relative—but not absolute— 
distance perception is some function of the ret- 
inal size of familiar objects. 


C. Perception of Distance 


The first time a reduced-cue situation was 
presented to a subject, the situation was a 
unique presentation in that the subject probably 
had never previously encountered a situation 
identical to it. But the subject had encountered 
normalsized playing cards prior to the experi- 
ment. (This prior experience is obviously neces- 
sary in order for the playing card to be a fa- 
miliar object) Since these prior ‘experiential 
situations were usually full-cue situations, thc 
absolute distance of the object was undoubtedly 
perceived. It might be expected that, upon being 
presented with a particular retinal subtense of 
a playing card in a reduced-cue situation, the 
subject would make a distance judgment rela 
tive to this prior experience. If such a result 
had been demonstrated, it would have been 
difficult to decide whether the distance judgment 
should be termed absolute or relative, Perhaps 
the reason that such a result did not appear in 
this study is that too long an average time in- 
terval occurred between the prior experience and 
the experiment. It will be recalled that, with the 
relative localization. demonstrated in the re. 
duced-cue situations, only a four-minute interval 
separated the successive presentations. 

This emphasizes the difficulty in demonstrating 
absolute-distance perception without the possi- 
bility that relative-distance perception had pro- 
duced the results. But, as has been indicated by 
this study, a relative-distance perception as a 
function of relative retinal size can occur inde- 
pendently of the occurrence of absolute-distance 
perception. И 

This study does not suggest that retinal size 
has no part in the perception of absolute dis- 
tance. In many situations, the perception of abso- 
Jute distance may at least partially consist of a 
series of successive relative-distance perceptions. 
In this sense, a series of relative retinal sizes 
may be a significant factor in determining the 
perception of absolute distance. Also, there is 
no evidence that, given the proper training, the 
subjects could not have learned to use the abso- 
lute retinal size of a familiar object as a cue to 
determine its absolute distance. But, a distinc- 
tion can be made between an immediate distance 
perception and an inferential estimate of dis- 
tance (2, р. 342). Therefore, whether such learn- 
ing would result in the ability to make an in- 
ferential estimate of absolute distance or would 
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produce an absolute-distance perception would 
remain to be investigated. 


V. SUMMARY AND CONCLUSIONS 


An experiment was conducted to de- 
termine whether the retinal subtense 
of a familiar object could act as a de- 
terminer of its apparent absolute dis- 
tance. To test this question, it was re- 
quired that each subject be presented 
with only one object in an otherwise 
homogeneous field, and that different 
subjects be used with the different 
retinal sizes of the same familiar object. 
Apparent distance was measured by hav- 
ing the subjects throw darts to the ap- 
parent distance-position of a playing 
card. Neither the trajectory, nor the 
place of impact of the darts was visible to 
the subjects. Two kinds of situations 
were used. In one, called the reduced-cue 
situations, the, purpose was to eliminate 
all distance cues but retinal size. In the 


other, called the full-cue situations, а 
variety of distance cues were present. 
The results from the full-cue situations 
were used to change throwing scores 
from the reduced-cue situations to per 
ceived-distance scores. The analysis of the 
average differences in the scores between 
the first presentations of different re- 
duced-cue situations revealed no evidence 
for the presence of absolute-distance per- 
ception as a function of retinal size. 
However, between the successively pre 
sented reduced-cue situations, a change 
in the magnitude of the scores as а соп" 
sequence of a change in retinal size did 
occur. This was interpreted to be the 
result of relative- not absolute-distance 
perception. It seems that relative-dis- 
tance perception as a function of the 
relative size of familiar objects can oc 
cur without the concomitant occurrence 
of absolute-distance perception. 
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I. THE PROBLEM! 


pue in study skills has been a 
part of the American college scene 
since the early 1920's. Since that time 
much effort has been expended develop- 
ing techniques of skill and evaluating 
their effectiveness, but little attention 
has been given to the role of the teacher 
in teaching the skills. The present in- 
vestigation is an attempt to analyze sev- 
eral methods of instruction as they relate 
to student outcomes in a how-to-study 
course. 


Two broad classifications of outcomes are 
considered, The first is termed “achievement 
Outcomes," and it is defined in terms of the 
specific learnings which take place during the 
course of instruction. Тһе learnings involve 
specific knowledge about effective methods of 
study, knowledge of research studies and their 
outcomes that cast light on study 5 Ils, knowl- 
edge of human characteristics that may help or 
hinder studying, and specific facts about reading. 
writing, outlining. note-making, testing, and 
concentrating that should help in study prob- 
lems. Knowledge of this nature is important in 
gaining an understanding of the purpose and 
techniques of effective study, but in and of it- 
Self does not guarantee practical application of 
the techniques learned. Therefore, it is also im- 
portant to measure the extent to which students 
adopt the methods they learn into their own 
Study habits, and the extent to which they form 
attitudes conducive to good study. Thus, knowl- 
edge and practice аге two indispensable measures 
of student achievement in a how-to-study course. 

The second classification of outcomes may be 
termed the “motivational outcomes"—an area 
~ 
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which has been virtually ignored in experi- 
mental studies on the outcomes of teaching. Edu- 
cators have not been unaware of the problem 
of motivation; the educational literature con- 
tains many references to the importance of 
motivating students. Many techniques and de- 
vices have been suggested in an attempt to raise 
the motivational level of students, However, al- 
most no experimentation has been done to de- 
termine the actual value of the suggestions. For 
the purposes of this study, one particular motive 
is under consideration—the need to achieve. Need 
to achieve may be considered an internal 
drive toward a specific action. directed. toward 
achieving what is called "success" in the Ameri- 
can culture, Achieving success involves being 
able to gain the recognition of one's fellow man; 
it involves striving to compete against some 
standard of excellence. In short, it may be called 
"drive" or "ambition." The achievement motive 
is not confined to one field of endeavor. The 
actual task is unimportant; it is the individual's 
attitude or internal state of affairs. impelling 
to be successful which is important. 


him 
Purpose 

The primary purpose of the present 
investigation is to study both the achieve- 
ment outcomes and the motivational out- 
comes associated with different methods 
of teaching in a how-to-study course. We 
are concerned with identifying instruc- 
tional methods that are most effective 
in improving the students’ knowledge of 
effective study methods, in encouraging 
the actual application of these methods, 
and in increasing the students’ level of 
achievement motivation. 

Two important subsidiary problems 
involve the question of whether the 
-pethod that is best for one type of stu- 
dent is also best for another. First, what 
influence does preference for a certain 
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type of instruction have on achievement 
and motivational outcomes? For ex- 
ample, do students achieve better when 
they receive the type of instruction they 
prefer, or do they perhaps learn more 
under an instructional method they do 
not like? Second, what relationship ex- 
ists between student ability-level and 
type of instruction received? For ex- 
ample, do bright students achieve better 
under one type of instruction while the 
less bright students improve more under 
another teaching method? The various 
interactions between teaching method 
and student preference for method, and 
between teaching method and student 
abilitylevel are tested in this investiga- 


tion. The specific null hypotheses are 
outlined in Part II. 


Unique Aspects of the Study 


Birney and McKeachie (2), in a 1955 review 
of the literature on teaching methods in psy- 
chology, defined two principal types of teaching 
methods—an instructor-centered and a student. 
centered method. Most of the studies cited by 
Birney and McKeachie consist of a comparison 
of the effectiveness of these two methods, Why 
is it necessary to use only these extreme methods? 
Is it not conceivable that a combination of 
methods might be more effective than strict ad- 
herence to any one? The present experiment 
makes use of an eclectic teaching method, an 
attempt to combine the advantages of both of 
the extreme methods. At the same time an 
instructor-centered and а student-centered 
method are also compared. The methods are 
defined in Part III. 

Another important feature of this investigation 
is that it measures motivational outcomes in 
addition to achievement outcomes. To the best 
of the authors’ knowledge, no studies have yet 
been performed measuring directly motivational 
outcomes associated with different teaching 
methods. The reason for this lack has probably 
been that adequate measures of motivation have 
not been available. It is only within the last 
few years that relatively valid measures of moti- 
vation have been developed. The measures used 
in the present investigation are described in 
Part IV. я 

The design of the experiment constitutes an 
important improvement over the design of many 
other studies in this area. Farquhar (6) has criti- 


TP 


cally reviewed the literature on teaching methods 
and has pointed out many of the methodological 
faults underlying previous experiments. The 
present investigation seeks to overcome many of 
the design limitations of previous studies. in 
order to secure more defensible interpretations 
of results. The details of the design аге outlined 
in Part II. 


П. THE DESIGN OF THE EXPERIMENT 


The study is designed to investigate 
differences in achievement and motiva- 
tional outcomes which may be associated 
with three teaching methods. As a self- 
contained experiment, it embodies three 
principles essential to modern research: 
randomization, replication, and control 


(12). 
The Basic Plan 


The teaching methods used in this 
study are instructor-centered, student- 
centered, and eclectic. Two instructors 
at two different times teach six sections 
of the how-to-study course using all three 
methods. Instructor A teaches by all 
three methods during the morning; 11 
structor B teaches by all three methods 
during the afternoon hours. The relative 
contribution of the different times and 
the different instructors to experimental 
Outcomes cannot be separated wie 
the limits of this design, The combined 
effect will henceforth be referred to û 
time-instructor, 

The design of the experiment is SUP 
marized in Table 1. 


TABLE 1 ў 
SUMMARY OF THE Basic EXPERIMENTAL DESIG 


sat Method 
i Time- uident. 
nstructor |Instructor- i tuce 
Centered Eclectic Centere 
Instructor A: ‘on 3 
Morning Section т Section 2 Sectio 


Instructor B: 


ion 6 
Afternoon | Section 4 Section 5 Section 


CR 
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THREE TEACHING METHODS 


Complete randomization was not possible be- 
cause of administrative difficulties. Students were 
randomly placed among the three teaching 
methods, but not between time-instructors. It is 
necessary to isolate the differences between the 
effect of time-instructors in the analysis, but this 
effect is not central to the purpose of the study. 
The primary problem is to detect differences in 
outcomes among methods, This randomization 
procedure provides the basis for obtaining à 
valid estimate of experimental error so that a 
probability statement about method differences 
based on a known distribution can be made. 

Replication is accomplished by having each 
method taught by two instructors rather than 
опе. This not only permits а valid estimate of 
experimental error, but also tends to diminish 
experimental error. 

Each method provides 
others, No attempt is mac 
which is given no instructior 
No one section is designated 
tion or as the experimental section. Each sec- 
tion is both a contro] and an experimental 
section in relation to each of the other sections. 


a control on cach of the 
le to provide a section 
1 in how to study. 
as the control sec- 


Primary Null Hypotheses 
sures are to be used 
in the main analysis and a complete 
description of these instruments is found 
in Part IV, With these measures the fol- 
lowing null hypotheses are to be tested: 
1. No differences exist among mean 
scores of the criterion instruments аб 
tributable to different methods of teach- 
ing. 
2. No differences exist b 
scores of the criterion instruments at 


tributable to different time-instructors. 
3. No interaction effect exists between 
teaching method and time-instructor. 


Subsidiary Null Hypotheses 


the many possible subsidiary 
ut of a study of this 


Four criterion mea 


etween mean 


From 
problems growing O 
nature, sies: to be especially perti- 
nent. One involves an attempt 10 dis- 
cover relationships between preference 
for certain instructional methods anc 
different outcomes associated with 4 
actual teaching by these methods. s 
Other is concerned with the relationship 
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of students' ability level to different out- 
comes of these methods. 

The following procedure is utilized in 
making the first subsidiary analysis: The 
students taught by identical methods 
(though by different instructors) are 
pooled and then split according to their 
preference for certain teaching methods. 
Preference is measured by the Preferred 
Instructor Characteristics Scale (PICS) 
which is fully described in Part IV. The 
design is represented in Table 2. 

The following subsidiary null hypo- 
theses are to be tested: 

1. No differences exist among mean 
scores of the criterion instruments at- 
tributable to different instructional meth- 
ods. 

о. No differences exist among mean 
scores of the criterion instruments at- 
tributable to preference for teaching 
methods. 

3. No interaction effect exists between 
teaching method and preference for in- 
structional method. 

The main concern of this design is 
with the third hypothesis—the test for 
interaction. From this hypothesis it will 
be possible to determine whether stu- 
dents with a certain preference are more 
motivated by one instructional tech- 
nique than another. 

А similar procedure is followed in an- 
alyzing ability-level relationships. The 
students taught by identical methods are 


TABLE 2 


SUMMARY OF DESIGN FOR SUBSIDIARY ANALYSIS 
ОЕ PREFERENCE FOR INSTRUCTIONAL METHOD 


Actual Method of Instruction 


Preference 


for: Instructor-| q " Student- 
Centered | Eclectic | Centered 
i 
Instructor- | (Scores on each of the criterion 
Centered | tests are entered in the nine cells 
Electic of this table.) 
Student- 
Centered | 


TABLE 3 


SUMMARY OF DESIGN FOR SUBSIDIARY ANALYSIS 
ON AniLITY LEVEL 


Actual Method of Instruction 


Ability 1 re, an — 
Level nstructor- РАШ? Student- 
Centered | Eclectic | Centered 
High | | | 
Third | (Scores on each of the criterion 
Medium tests are entered in the nine cells 
Third | of this table.) 
Low | 
Third | 


again pooled and this time split into 
thirds on the basis of their scholastic ap- 
titude level as measured by the American 
Council on Education Psychological Ex- 
amination (ACE). The design is sum- 
marized in Table з. 

The following null hypotheses are to 
be tested: 

1. No differences exist among mean 
scores of the criterion instruments at- 
tributable to different teaching methods. 

2. No differences exist among mean 
scores of the criterion instruments. at- 
tributable to different ability levels. 

3. No interaction effect exists between 
teaching method and ability level. 

Here, as in the first subsidiary analysis, 
the primary concern is with the inter- 
action hypothesis-the indication of 
whether students of certain ability-levels 
are more affected by one instructional 
technique than another, 


The procedure of pooling students taught by 
identical methods but different time-instructors 
is justifiable in the subsidiary analysis if there 
are no significant time-instructor differences and 
no significant interaction between time-instruc- 
tor and method in the primary analysis. If the 
preceding conditions are not met in the primary 
analysis, the data can be analyzed by a three- 
way analysis of variance and covariance, isolat- 
ing the effects of time-instructor, method, and 
ability level or preference for method variables. 


Statistical Analysis 


Analysis of variance and analysis of 
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covariance are the fundamental statistical 
tools used in the present investigation. It 
is possible by the use of these tools to 
isolate the sum of the squares associated 
with each experimental variable and test 
its significance statistically. | 

Certain assumptions are involved in 
the use of these tools. 


1. The observations should be normally dis- 
tributed within cach cell. This assumption of 
normality is seldom tested. As Johnson states, 
“This assumption, however, is not especially 
important” (12). It is not tested in the present 
investigation. : 

2. Variance within cells should be equal. Tus 
is an important assumption and is tested by the 
L, Test (sce reference 12). 

3. Observations should be randomly selected. 
Details of the present. randomization procedure 
are found in Part III. -i 

4. In covariance problems the linear 2 e 
sion coefficients of posttest scores on рее 
Scores should be equal within cells, The test a 
this assumption is provided by a test of es 
homogeneity of regression coefficients (scc refer 
ence 10). 


Each of the four criterion tests is at 
ministered before and after the instruc 
tional period. The results of the first 
administration are hereafter referred tO 
"c ^j S 8 
as pretest scores and the results of th 
second administration as posttest scores- 

, H H H 1 1 И j st 
By this technique, it is possible to adjus 
statistically differences in posttest score 
“a ee . H scores 
for initial differences in pretest score 
through analysis of covariance. 

А 11‏ ت 
The number of students falling in cach с‏ _ 
not equal. Therefore, a special computation?‏ 5 
technique is required in a two-way analysis d‏ 
variance, if an exact test of significance is i‏ 
Such an exact test, involy ing a least-square апат‏ 
Sis solved by pivotal condensation (4), is €‏ 
ployed in the present investigation. 2‏ 

The level of Significance for rejecting 
primary and subsidiary null hypotheses ЕА 
arbitrarily set at the five per cent level, For tes™ 
of assumptions the one per cent level is set. 


the 
is 


TIT. EXPERIMENTAL PROCEDURES 


T! á . :ning 

This part is concerned with outlining 
in detail the experimental progowe 
specified in the design. The selection О 


© 


=} 
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а random sample, the specific definitions 
of the instructional situation and meth- 
ods, and the various checks on the quali- 
tative differences among methods and the 
consistency of instruction between in- 
structors are developed in this order. 


Selection of the Sample 


One assumption underlying the use of 
the statistical tools of this investigation 
is randomness of observations. Before the 
experiment actually started, it was there- 
fore necessary to specify the population 
accurately, and to devise а procedure to 
assign this population randomly to the 
three teaching methods under investiga- 
поп. 


The Population 


The population for this study consists 


of all University of Minnesota students 


who elected to take Personal Orientation 


1, How to Study, in the winter quarter of 
1955. This population consists of both 
male and female students from five col- 
leges of the University: the College of 
Science, Literature, and the Arts, the In- 
stitute of Agriculture, the Institute of 
‘Technology, the College of Education, 
and the General College. Most of these 
students are freshmen and sophomores, 


but a few are upperclassmen. 
population be con- 


In no way can this t " 
mple of the entire 


sidered a random sa 4 e 
college and university population 
United States, nor can it be consider 


$ iver: of 
even representative of the University 


Minnesota. Students enroll in a how-to- 
study course for a variety of reasons: Ns 
feel that they need help with their re 
problems and are motivated to he poini 
where they want to take action. ur 
clusions about this population Frei 
fiably be generalized to other 918511: 


Populations. 
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cr 


Randomization Procedure 


Students desiring to register for Per- 
sonal Orientation 1, How to Study, given 
in the winter quarter of 1955, were re- 
quired to have the written consent of 
the instructor. Obtaining permission of 
the instructor is not the usual procedure 
with this course, but it was considered 
necessary as an aid in the randomization 
procedure, 

The purpose of the randomization procedure 
was to place students at random in one of the 
three teaching methods. No attempt was made 
to place students randomly between time-instruc- 
tors. The procedure was designed so that students 
could have their choice of a morning or after- 
noon class and the corresponding instructor. But 
once that decision was reached, the one section 
out of the three possible sections of that time- 
instructor was determined randomly. 

When a student appeared for registration, he 
was given a choice of two sections which had 
been predetermined in a random manner except 
that one was always from the morning sections 
and the other from the afternoon sections. If he 
were able to accept one of these two sections, 
he was entered as a random student for that 
section. The next student to appear for registra- 
tion was given his choice of the next pair of 
randomly determined section numbers, 

If, for any one of a number of acceptable 
reasons, a student was unable to work the 
randomly assigned section into his schedule, he 
was allowed to sign up for the section he desired, 
but was designated a nonrandom student in that 
section. The next student to appear was then 
iven the choice of the same two section numbers 
had been unacceptable to the previous 


8 
which 
student. 

The procedure continued until one of the 
sections was filled to capacity (30 students). 
Thereafter, no more random assignments were 
made in the remaining two sections for that 
time-instructor. The succeeding registrants in 
those two sections were all designated as non- 
random. Random assignment continued in the 
other time-instructor sections until one of these 
sections Was filled, All of the remaining regis- 
trants were then designated as nonrandom, At 
the end of the procedure 192 students had 
registered. Of these, 121, ог 63 per cent, were 
1domly assigned. The remaining 7! students, 


р 
were nonrandomly assigned. 


or 37 per cent, 


Characteristics of the Population 
quarter in which instruc- 


During the winter 
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TABLE 4 
NUMBER ОЕ STUDENTS FALLING IN EACH COLLEGE CLASS AND SECTION 
Section 
Class шой Time-Instructor A Time-Instructor B Total 

I 2 3 4 a 6 

Freshmen Random 13 16 17 13 17 8 Уч 
Nonrandom 6 6 9 8 4 8 4 

Sophomores Random 6 3 3 2 3 6 13 

Nonrandom I 2 2 2 3 7 
Juniors Random 1 o 1 2 I 5 
Nonrandom o o o 2 o 4 
Seniors Random o o o o o o с 
Nonrandom 2 o o 1 o o 3 
Total Random | 20 19 21 17 20 15 pod 
Nonrandom 9 8 II 13 9 15 5 


tion took place, fifteen students dropped from 
the course. No particular disproportion appears 
among the sections, between time-instructors, or 
between the random and nonrandom groups. It 
seems safe to conclude that the distribution of 
dropouts introduces little bias into the results, 

The remaining students Consist of 119 ran- 
dom and 65 nonrandom students, The distribu- 
tion of these students by college class and by 
section appears in Table 4. About 75 per cent 
of the randomly placed students are freshmen, 
21 per cent are sophomores, 4 per cent are 
juniors, and zero per cent are seniors. In gen- 
eral, the placement of the nonrandom students 
was similar to that of the random. 

The proportion of random students to the 
total enrollment in each of the six sections 
tends to be approximately the same, The range 
is from 50 per cent in section 6 to 70 per cent 
in section 2. The proportion of random students 


in the entire group is 63 per cent. 

Distribution бг наа by sex is рете) 
in Table 5. It is evident that the proportion О 
cach sex within sections and between mca 
and nonrandom groups remains reasonably ч з 
stant, However, as an additional check on ы, 
factor, separate analyses are reported in vont Ne 
for the males only and for the males and fema 
combined, А 

The distribution of students by college 18 
summarized in Table 6, About 75 per cent ЧЕ 
the randomly assigned students are registered і 
the College of Science, Literature, and the 35 
12 per cent in the College of Education, 3 ae 
cent in the Institute of Technology, 4 рег д, 
in the Institute of Agriculture, and 6 per E 
in the General College. The proportion of no is 
random students from each of these colleges ® 


ree тай ies in- 
similar, considering the small frequencies 
volved, 


TABLE ; 
NUMBER oF STUDENTS OF EACH Sex IN Елси SECTION 
Section 
Random or ч » 
Sex Nonrandom: Time-Instructor A Time-Instructor B. Total 
I 2 3 4 5 6 

Male Random 17 16 19 10 12 9 83 
Nonrandom 7 g 9 тї 7 12 53 
dis Random 3 3 2 6 29 
Fem Nonrandom 2 т 2 A d 3 22 
Total Male 24 23 28 21 19 21 136 
Female 5 4 4 то 4 
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TABLE 6 
= NUMBER OF STUDENTS FROM Eacn COLLEGE OF THE UNIVERSITY, BY SECTION 
Section 
, | к= Random Ti : 
College NR = Nonrandom Time-Instructor A | Time-Instructor B Total 
| т a 3 | ë 3 6 
۴8 7 j 
science, Literature, R 14 14 19 II 14 12 84 
nd Arts NR 6 т me Ж 8 13 55 
Education R 1 3 o 5 3 1 13 
NR 2 o o І I I 5 
Institute of Tech- R | 5 o 1 o o 1 3 
nology NR o 5 o o I o 1 
Institute of Agricul- R а 9 д = ы 9 5 
ture NR o a e o 9 I i 
General College R 2 2 ы 2 = 7 
NR I 1 o o o 3 
Total R 20 19 21 17 20 15 112 
NR 9 8 ІІ 13 9 15 65 
An accurate attendance record was kept absences were few enough in any section and 
throughout the quarter. It was important to relatively constant enough among sections to 
check this factor in order to ascertain whether justify the conclusion that this factor introduces 
the students from different sections had 2n equal little bias into the results. 
exposure to the instructional method. Out ofa Do the sections vary in terms of their average 
total of 29 class periods during the quarter, the scholastic aptitude? Records of the American 
class Council on Education Psychological Examina- 


average of 2.07 
hile the nonrandom 
2.72 class periods, 


random students missed an 
periods, or 7.1 per cent, W 
students missed an average of 2-72 
Or 9.4 per cent. The average number of class 
periods missed for the random students of each 
section remained relatively constant over all six 
sections, The range Was from 1.65 class periods 
(5.7 per cent) in sections 4 and 5 to 2.79 class 
periods missed (9.6 per cent) in section 2. The 


TABL 


CORE ON 


THE AMERICAN Co 


ere obtained from the Student 
Counseling Bureau. Scores on different forms of 
this examination were converted to equivalent 
raw scores on the 1952 form by means of con- 
version tables provided by the Student Counsel- 
ing Bureau. A summary of the mean raw scores 
of random students for each section appears in 


Table 7. 


tion (ACE) w 


E7 
UNCIL ON EDUCATION 
ЕАСН SECTION 


EA wS 
> PA снооб1сА, EXAMINATION FOR 1 
RANDOMLY PLACED STUDENTS ONLY 
Method 
ae i Student- Total 
Sex Instructor нен) Eclectic soe, 
/ 8.7. 107-95 103.86 104.05 
Males and Females Ё EX 15 08.05" 94.87 96.69% 
шы Total 97-83 102.87" 100.11 о вна 
.76 111.06 103.32 Sm 
RON NO В 99-5 99.59% 80.80 95.23% 
B тоа! 98.08 106.11 99.09 101.13 
i „оша be expected because no ACE 
i IIs is one less than woul score 
* The number of students in these € male 9 Section 5. 
was available for one гап omly pla! -2 | 
= = w, Research 


Bu? 


\ 
piu : COLLEGE | 


" 


[= 

И 

И 
i 
\ 
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An examination of Table 7 reveals a tendency 
for the sections of time-instructor A to score 
higher than the sections of time-instructor B, 
while scores in sections taught by the same 
method seem to be relatively more homogeneous. 
An exact test of significance using analysis of 
variance was made to investigate the statistical 
significance of these differences. 

For the males and females combined, dif- 
ferences in mean scores between time-instructors 
and among methods do not reach significance. 
However, when males only are analyzed, the 
difference between time-instructors becomes sig- 
nificant at the five per cent level, but differences 
among methods remain insignificant, The inter- 
actions are not significant in either case. 

This significant difference between time- 
instructors could possibly be a result of the 
manner of randomization. Students were given a 
choice of a certain time-instructor but were 
randomly assigned to methods. Possibly some 
selective. factor operated in making this choice 
to create the significant difference. The im- 
portant fact, however, is that no significant dif- 
ferences exist among methods, Apparently the 
randomization procedure was successful in dis- 
tributing students among the three methods so 
that approximately equal means and variances 
resulted, 

То compare the average ACE. standing of the 
random and nonrandom students, Student's. | 
test, as reported by Johnson (12, 
The 61 nonrandom students or 
scores are available have a mean raw score of 
103.23; the 111 random students have a mean 
of 100.34. The variances are approximately equal 
(Е = 1.35). The resulting t value is .917 and is 
nonsignificant. It may be concluded that the two 
groups are approximately equal ir 
ability. 


р. 72), is used. 
п whom ACE 


1 scholastic 


The Nonrandom Students 


The 65 students who could not b 
assigned are not included in 
results. The data presented in the preceding 
section show that they are not essentially dif- 
ferent from the randomly placed. students on 
any of the characteristics examined, However, 
it is impossible to examine the differences be- 
tween the two groups on every variable that 
might have any relevance, and no attempt in 
that direction has been made. 

In order to meet the assumptions of (he 
statistical tools, it was judged best to confine the 
analysis only to the randomly placed Students, 
The possible influence of the nonrandom stu- 
dents on the random students in each section has 
been considered, but the subjective impression 
of the instructors is that they exerted no in- 
fluence out of proportion to their number, The 
students themselves did not know whether they 


€ randomly 
the analysis of 


had been randomly or nonrandomly assigned, 
and this concept was never discussed with them. 


The Instructional Situation 


For a more complete understanding of 
the experiment, it is important to con: 
sider the setting, the particular course 1n- 
volved, and the instructors themselves. 
This section is devoted to a brief descrip- 
tion of these factors. 


The Setting 


Al classes met in. Room 307, Johnston Hall, 
on the University of Minnesota Minneapolis em 
pus. The room was equipped with movable tables 
and chairs, which were ordinarily placed in он: 
large circle, The room was also equipped with 
black shades so that it could be darkened fo! 


[ vi ; ^ > length 
use of visual aids, A blackboard ran the leng 
of one wall, 


The Course 


Personal Orientation à, How to Study, is | 
two-credit, one quarter course. Classes meet P 
Mondays, Wednesdays, and Fridays during d 
ten-week quarter, Administratively it is po 
in the General Studies Department of the Col- 
lege of Science, Literature, and the Arts. Дн 
open to students from other colleges, and there 
are no prerequisites, ze dn 

The college bulletin describes the course 
the following terms: ` 

“Practical assistance to the student in develop 
ing efficient methods of study and concentration 
organizing material, preparing for examinati 
and improving reading ability, Attention to ! i 
orientation of studen in their attitudes is 
motivation, and the relation of these to satis: 
factory performance" (19, p. 74). 7). 

Two textbooks were used in the course (1, the 
The topics covered by the texts and by з 
course include such things as problems of Hes 
ing, Concentration, study methods, use of um. 
note making, writing, 1 
aminations, 


and preparation for 


The Instructors 


Both instructors in the study were =o 
candidates, with majors in educational psgchicloms 
апа minors in psychology. Both were using Г 
Same population and the same experiment 
Procedures with different variables for te! 
doctoral dissertations, jon 
Instructor A had taught Personal пешинин 
for one quarter prior to the experiment. 15 
addition, he had had two years of previo" 
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leaching and counseling experience at the high- 
school level. 

Instructor B had taught Personal Orientation 
1 for five previous quarters. He also had previous 
teaching experience on the college level. 

Both instructors had had an opportunity to 
try out the different teaching methods before the 
start of the present experiment. In fact, it was 
the trial and error and experience with these 
different teaching methods that lead the instruc- 
tors to become curious about which method 
might be superior. 


The Instructional Methods 


‘Three different methods of instruction are 
utilized in the present investigation, These in- 
clude the instructor-centered method, the stu- 
dent-centered method, and the eclectic method. 


Instructor-Gentered 


The instructor-centered teaching approach fol- 
lows the line of the traditional lecture method. 
The philosophy of this method is that the in- 
structor is the authority and his task is to con- 
vey information to students so that they may 
learn and apply. 

After all pretest measures had been ad- 
ministered and prior to the beginning of the 
actual instruction, a syllabus was handed to the 
students, containing, in addition to general 
course information, specific reading as meus 
The exact number of pages tO be read 2a ai 
textbook prior to cach cla period was specified 
In this syllabus. 

No attempt wa 
comments or questions in the 
method, The instructor beg 
calling roll, and then he proc 
т а; оп occasion, а student 
запа to ask a question, the insta 3 
occi crt’ ا‎ specifically. The ee 
did not ask for student opinion oF Tr p "o 
10 share their experiences on Шеш шу m en 
lems, When a students question han "n is 
answered, the instructor would continue MIS 
€cture or other activity. 

The eere maintained а formal but s 
a rigid attitude, Humor Was occasionally in re- 
duced when appropriate. The instructor 
mained standing at all times. 

The students were told to epe 
nounced quizzes over reading macy students 
Quizzes were exchanged and score ri rs, €X- 
in class, The instructor read the anwe E 
Plained why each answer was O ав 
answered any remaining questions ай 


direct 

y. "= lepended 
Students were told that thelr grade o 
Upon their test scores from а final aot quizzes 
а midquarter examination, and the $ 


made to encourage student 
instructor-centered 


an cach period by 
седеа to lecture. 
would raise his 
instructor woul 


unan- 
These 
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In addition to lecturing, the instructor oc- 
casionally lead students in a period of practice 
on a certain study technique, All such practice 
exercises were done at the direction of the in- 
structor at the time and in the manner pre- 
scribed by the instructor, 


Student-Centered 


The student-centered teaching method fol- 
lowed in a general manner the method set forth 
by Flanders (7). The philosophy underlying 
this method is that a group of students has 
within itself the ability to solve its own problems 
under proper leadership. Since the how-to- 
study course is organized more to solving prob- 
lems than toward conveying content, Flanders 
technique seems ideally suited for testing the 
student-centered method. 

After the pretests had been administered, the 
students were handed a syllabus with a state- 
ment of course objectives, fundamental considera- 
tions, names of the textbooks, and a list of 
personnel services available at the University. 
No mention of assignments was made in the 
syllabus. 

Each student was given a name card to place 
bcfore him on the table, and the informality of 
the class was emphasized by having each student 
introduce himself to the class. The instructor 
lead the class in a discussion of the ways in 
which a group operates, how it succeeds, and 
what may cause it to fail. He emphasized that 
the class belonged to the students, and they 
were responsible for its operation. ` 

The students were divided into groups of 
six or less to talk over their study problems and 
compile a list of problems they felt were most 
important for them, These problems were listed 
on the board and organized into topics. The 
instructor told the class what parts of the text- 
s related to each problem they had sug- 


book: 


gested. me 
ch student was asked to indicate the topics 


ich he would be most interested in work- 
class was divided into committees on 
the basis of these choices. The committees were 
structured by the instructor to a limited extent. 
They were asked to elect a chairman and a rc- 
corder and were held responsible for investigat- 
ing their topic, reporting their findings to the 
class, and presenting some class activity in which 
the entire class would participate. 

At this point, the instructor assumed the role 
of a resource person. He circulated among the 
committees, answering questions, helping to pro- 
vide information, and making suggestions when 
called upon. However, he attempted, as far as 
possible, to reflect questions in order to place 
primary responsibility upon the students them- 


selves. 
Wh 


on wh 
ing. The 


en the planning was completed, each com- 
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mittee presented its report in turn. The reports 
were presented in such a manner as to en- 
courage class discussion. During the reports the 
instructor was seated among the students, and 
would interject ideas and questions occasionally, 
much as the students felt free to do. 

Activities and reports varied widely. They 
included such things as panel discussions, open 
class discussions, short lectures, role-playing, 
movies, quizzes, reading tests, impromptu themes, 
surveys, use of the opaque projector, and prac- 
tice on study skills. 

The students were told that their grade would 
depend upon a final examination, a midquarter 
examination, any quizzes devised by the com- 
mittees, and upon the quality of their class 
participation. 


Eclectic 


The eclectic method did not confine itself to 
any one manner of presentation. А combination 
of many instructional techniques was used. These 
included many of the techniques used in the 
two methods previously described. 

The class syllabus contained the same informa- 
tion as the student-centered syllabus, plus а list- 
ing of the main topics to be covered in the 
course, The readings relating to each topic were 
also given. 

Name cards and introductions were used to 
help the group get acquainted and establish 
a relatively informal atmosphere, A variety of 
instructional techniques was used with this 
group. The essential difference between the 
eclectic and student-centered methods was that 
in the former the instructor suggested the topics 
to be covered and the manner in which they 
were to be presented, The eclectic method 
differed from the instructor-centered method in 
that a maximum of student participation was 
encouraged, although occasional lectures were 
interspersed with other activities, 

The primary technique used was class discus- 
sion. Different devices were used to get a maxi- 
mum of group participation. The buzz session 
was a technique whereby students would be 

given a question posed by the instructor and 
would be asked to discuss it with fellow students 
sitting near them. This served to warm the stu- 
dents to a general class discussion, The six-by- 
six technique was a more formal type of buzz 
session. Groups of six or less students would be 
formed to discuss a question for six minutes 
and then report their conclusions back to the 
class. 

The students were told that their grades 
would depend upon a final examination, а 
midquarter examination, unannounced quizzes, 
and the quality of their class participation. 


Consistency of Instruction 


One of the essential features in the de- 
sign of this experiment is the principle 
of replication. To implement this prin- 
ciple it is important that the three treat- 
ment methods be consistently carried out 
as they are intended to be. Three tech- 
niques were used to check on the charac- 
teristics and consistency of the instruc- 
tion: coordinated planning by the in- 
structors, a survey administered to the 
students, and ratings of class recordings. 


Coordinated Planning 


The two instructors planned in detail the 
specific content to be covered and the specific 
instructional techniques to be used in both the 
instructor-centered and eclectic methods, This 
was also done for the first part of the student- 
centered method, when a certain amount of 
class structuring was necessary. 

The coordinated planning was concerned not 
only with long-range goals but also with the 
specific daily plans. Before cach day's teaching, 
the instructors prepared their activities jointly. 
In addition, instructor B, who taught the after- 
noon sections, routinely checked with instructor 
A at noon to ascertain whether the instruction 
had proceeded as planned. When for one reason 
or another the material covered was not exactly 
as planned, or instructor A had found it neces- 
sary to use some other technique, Instructor 
altered his plans so as to conform as nearly 9? 
Possible to what had occurred in the morning: 

After the initial structuring in the gader 
centered groups, little instructor planning cou = 
occur. The groups had been taken over by А 
students and further instructor planning war 
not only unnecessary but unwise, However, t 
instructors did compare notes on the role m 
assumed in class to keep it as consistent 
student-centered as possible. 


Survey of Opinions—Behavior (soo-B) 


The SOO-B was devised by the instructors t° 


discover whether students perceived the instrue 
tion in the way it was intended. Eight items 
were derived from the definitions of the teach- 
ing methods. These are items 11-18 in the pA 
vey of Opinions, a copy of which is found in th 
Appendix. The first ten items make up SOO 
which is discussed in Part ТУ. f 

Arbitrary weights were assigned to cach © 
the five possible responses of cach item. 
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weight of four was assigned to a response judged 
most representative of the instructor-centered 
method, and a weight of zero to a response most 
characteristic of the student-centered method. 
Intermediate weights of three, two, and one 
Were assigned to corresponding intermediate re- 
sponses. Thus, with these arbitrary weights, the 
highest possible score, indicating perception of 
an instructor-centered method, would be thirty- 
two; the lowest possible score, indicating percep- 
tion of a student-centered method, would be 
zero, 

The SOO-B was first administered at the end 
of the third week of instruction to the 166 
random and nonrandom students present that 
day, Using these responses, à new series of 
Weights was devised using the reciprocal averages 
technique (15). In brief, the method involves a 
Series of successive approximations or iterations 
to obtain the item weights yielding a maximum 
interna] consistency. Five iterations were neces- 
sary to stabilize the item weights for the SOO-B. 
The reliability of the final iteration was .87. 

The SOO-B was administered first at the end 


of the third week, for the second time at the 
end of the sixth week, and for the third time 
т. For the 


during the final week of the quarte 
fist and second administrations all students 
Present, whether random Or nonrandom, were 
included in the analysis. An approximate method 
of calculating analysis of variance with unequal 
frequencies in cells was judged to be sufficient 
for analyzing this data (12, р. 261-266). For the 
third administration, only randomly placed stu- 
dents were used in the analysis and an exact test 
Of significance was used (4). Only the analysis 
of the third administration is presented here: 
The analyses of the first and second administra- 
tions yielded approximately the same results: 
The hypotheses tested in all three administra 
tions of the SOO-B are as follows: 

1. No differences exist between mean scores 
of the SOO-B attributable to different time- 


Instructors, 


IN A HOW-TO-STUDY COURSE 11 


TABLE 8 


Mean Scores or SOO-B ror EacH SECTION 
THIRD ADMINISTRATION 


N=111 Random Students 


Method 
Instructor Total 
Student- decens Instructor- otal 
Centered Eclectic Семегей 
A 7.0052 10.8947 22.0000 13.5667 
B 8.4667 9.5203 13.6471 
Total 7.0607 10.2105 13.6036 


2. No differences exist among mean scores of 
the SOO-B attributable to different teaching 
methods. 

3. No interaction effect exists between teach- 
ing method and time-instructor. 

For the third administration of the SOO-B 
the students were asked to base their responses 
on the entire quarter's work, One random stu- 
dent in section 5 missed this administration so 
the total number of students included js 111. 
The mean scores are reported in Table 8. 

A high score indicates perception of an in- 
structor-centered approach, and a low score 
indicates perception of a student-centered ap- 
proach. Inspection of Table 10 reveals that the 
mean scores аге highest for the instructor- 
centered sections, lowest for the student-centered 
sections, and intermediate for the eclectic sec- 
tions. This would tend to indicate that on the 
tudents perceived each class as the 
instructor intended to teach it, Furthermore, the 
two instructors were perceived to be operating 
similarly in each teaching method, 

To test the significance of these differences, an 
method of analysis of variance was cal- 
The analysis appears in Table о. This 
analysis shows that the differences among meth- 
ods are significant at far less than the one per 
cent level, while differences between time- 
instructors and the interaction are nonsignifi- 
cant. The three administrations of SOO-B all 


average 


exact 
culated. 


TABLE 9 


ANALYSIS 
SOO-B THIRD 


оғ VARIANCE 
ADMINISTRATION 


dom Students 


Nor Кап = 
; Hypothesi: 
f Mean p = [ур fs 
jati df еен Square Tested 
Source of Variation А qu _ red 
5 .1057 жр ссерї 
Aotween Ti Ү 1625 19230 164.07* Reject 
n Time-Instructors 879. 8460 2439 
[mong Methods 2 ы 25.1947 Hee 1.15 Accept 
Е eraction Y 1548-3398 ty don 
Resid 1 107 1582-5345 x 
ua 
Total 110 


* p«.or. 
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yielded the same general results, The students 
perceived each method as the instructors in- 
tended to teach it, Furthermore, the results 
indicate that the instructors were successful in 
coordinating their planning so that they con- 
sistently portrayed the definition of their roles 
during the entire quarter, 


Class Ratings 


As a further check on the consistency of in- 
struction, tape recordings were made of all six 
sections during one day of instruction. The 
recordings were analyzed in several different 
ways. First, ratings were made of the amount 
of time that the instructor talked in each section. 
Second, two experienced teachers were asked to 
listen to cach tape and identify which method of 
instruction was used, And finally, these two 
raters were asked to write a subjective descrip- 


tion of each class after listening to the 
recordings. 
Talk ratio. The tapes for all classes were 


played while raters tabulated at ten-second in- 
tervals whether a student or an instructor was 
talking. The tabulation was done by alternate 
ten-second intervals; that is, the first rater tallied 
at the end of 10, 20, 50, . . , seconds, while the 
second instructor tallied at the end of 5, 15, 
25, . . . seconds. The percentage of total time 
occupied by the talking of the instructor is in- 
dicated in Table 10, 

An inspection of this table shows that the 
highest percentage of instructor talk occurred 
in the instructor-centered sections and the lowest 
percentage in the student-centered. In the 
eclectic sections, both instructors talked about 
half the time. These results are in agreement 
with the general principles involved with each 
type of teaching. Of course it must be remem- 
bered that these percentages are based on only 
one day's instruction and not a necessarily typi- 
cal day in the judgment of the instructors, They 
do give a rough idea as to the differences in 
communication patterns among the 
methods. 

Identification of methods. The recordings were 


three 


TABLE 10 


PERCENTAGE OF TOTAL TIME OCCUPIED ny 
TALKING OF INSTRUCTOR ON ONE Day 


Method 
Instruc- |р, i. 1 

ү Rater Student- Exleetic Instructor- 
Centered ^ entered 

A Y 6.8 54.5 99.5 

R 4..5 57.6 99.5 

B 1 | 15.3 g20 84.6 

2 | 24.0 53-8 90.3 


presented to two other experienced teachers in 
order that they might identify the method used. 
These two raters were asked to match cach tape 
recording with one of the three methods. They 
were given the definitions of the three methods, 
and then listened to the complete tape record- 
ing. | 

At the end of each recording cach rater In- 
dicated on paper the method he thought had 
been utilized. Both were 
identification of all ‘Jass periods. Р 

Subjective description. In addition to making 
an identification of cach class, cach of the two 
raters was also asked to write a brief paragraph 
of description, The descriptions for each class 
period for cach rater are not included here, but 
the reader may find them in two sources (6, 
PP. 65-68; 13, pp. 65-68). 


raters correct in the 


Subjective Impressions 
Certain unscientific subjective impres- 
sions were formed by the instructors dur- 
ing the course of the experiment. Perhaps 
these impressions could best be expressed 
as answers to questions that have fre- 
quently arisen. 


1. Can one really change his teaching ie 
ods as quickly as was nec ary 10 perform pis 
experiment? Changing one's method. of instruc 
tion is not as difficult as it may seem, It очае 
some extra preparation but it сап be donc. for 
fact, one of the major subjective ошто y 
the instructors was that they discovered | rd 
could be more flexible than they had previous? 
thought possible. 

мА did the students react to the idea of 
being guinea pigs for the experiment? Actua n 
few if any students knew that an expe шаш e 
teaching methods was in progress. ‘They ¥ an 
told that the tests they took were part pi he 
evaluation study, This fact was made dlear : 
first day the class met. However, they did 3 
seem to be aware that their section was te Dn 
a different type of instruction than any g 
How-to-Study section, Apparently, students jaly 
cussed the activities of the class very little. Т 1s 
one student mentioned the fact that his syllabus 
was not like that of another student's from ^ 
different class, 10 

3. Does one Have trouble getting students ok 
cooperate in taking the tests? All testing ms 
place during the class time. The students ча 
Promised a copy of their test results at ve nts 
of their testing. These were mailed to SIU à 
who were interested enough to leave à pas 
addressed, stamped envelope. The students i 
most Cooperative throughout the expenmen 


ng 
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4. What sort of feelings does one have about 
cach of the methods? Certainly the instructor- 
centered method is the most “ego inflating” 
method from the point of view of the instructor. 
The students seem to have more “respect” for 
the instructor. They tend to address him as 
Mr.” or “Sir,” and frequently stop after class 
to ask the instructor's opinion on а certain study 
problem, At the end of the quarter there seemed 
to be more expressions of gratitude from stu- 
dents in the instructor-centered method. In the 
eclectic and student-centered methods there 
Was less of this particular brand of "respect." 
Students felt free to disagree with the instructor 
and less often stopped after class to ask ques: 
tions, There is no question in the instructo! s 
minds that they actually taught by three. dif- 
ferent methods. - 


IV. EVALUATION INSTRUMENTS 

The purpose of this part is to describe 
“ach of the tests and criterion measures 
used in the present study. ‘These include 


the final examination; the survey of 


Study Habits and Attitudes; the л 
Achievement test; the Opinion, Attitude, 
{ Opin- 


and Interest Survey; the Survey 0 
ions—Attitudes; the Preferred Instructor 
Characteristics Scale; the American 
Council on Education Psychological Ех 
amination; and the Personal Data card. 
The first four of these measures were 
administered to all students the first week 
of class before any instruction had been 
completed. The first ac 
hereinafter referred to 35 the j 
While the second administration 15 € 


the posttest. 


Iministration is 
pretest, 
alled 


Final Examination 


e final ех 


A gg-item objectiv 
is 7 the studen 


constructed to measure 
апа understanding, of o al examina- 
Process of standardization 


three quarter ie 
revised to eliminat 


The fourth and 


to-Study courses offered dur 
1954-1955. During the first 
examination was constantly 
the nondiscriminating items. AGE Ubo: 
final form was administered to a E to-Study 
Students from the fall. тей in regar 
course, Their responses Were y on, and 
to item difficulty, item d 

reliability. 


1054 
апа! і 
jiscriminal 
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The difficulty of the items was determined by 
the number of correct responses for each item. 
There were 46 items in the 25 to 75 per cent 
difficulty level, 50 above the 75 per cent level, 
and three below the 25 per cent level. The test 
was negatively skewed with regard to difficulty, 
with at least half of the items answered cor- 
rectly by 75 per cent of the students. 

Phi coefficients were calculated by using the 
upper and lower 27 per cent of the total dis- 
tribution of scores. Sixty-seven, or two-thirds, of 
the gg items correlated .20 or above with the 
total score, The remaining items fell in the 
chance area, None were reversal items, By means 
of Hoyt's (11) analysis of variance technique, 
the reliability coefficient was found to be .67. 
This is not as high as is desired for a criterion 
instrument. The relatively low reliability might 
have been a function of the restricted range of 
correct response due to lack of difficulty in the 
test, However, the reliability was judged suf- 
ficiently high to provide a reasonably accurate 
differentiation between individuals inasmuch as 
the F value "between individuals” in this analy- 
sis was significant at the one per cent level. A 
random sample of fifty students from the current 
experiment was used to provide a check on the 
reliability and an almost identical .68 was 


obtained. 


Survey of Study Habits and Attitudes 
(SSHA) 


The Survey of Study Habits and Attitudes 
(SSHA) was used to measure student changes in 
study habits and attitudes toward study, Brown 
and Holtzman (8) have reported most of the 
essential information about this measure in their 
The SSHA is empirically keyed for 
those items which discriminated between stu- 
dents of high and low grade-point average. Be- 
cause a SCX difference was found in the predic- 
tive power of the items, the final form was scored 
on 36 items for men and 29 items for women. 
In the present investigation, the analysis of 
results for the SSHA is restricted to males be- 
cause of the limited number of females. 

The internal consistency reliability of the 
SSHA was estimated by Brown and Holtzman 
by the split-third method. For men, the Spear- 
man-Brown prophecy formula estimated the 
reliability to be .93. The test-retest reliability 
after а two-week interval was .95- On other 
samples with an eleven-week interval between 
administrations the estimate was .88. 

Estimates of the validity of the SSHA as a 
predictor of one-semester grade-point averages 
were obtained from samples of ten different uni- 
versities across the country. The validity co- 

27 to .66. All correlations 


efficients ranged from 
4 to be positive and significantly dif- 


zero for all schools. 


manual. 


were foun 
ferent from 
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The correlation between SSHA and ACE was 
found to be consistently lower than the correla- 
tions between either of these measures with 
grades. An exact test of the contribution of the 
SSHA to the prediction of grade-point average 
over and above that made by the ACE would 
utilize the multiple-regression technique. How- 
ever, this was not reported, so the interpreta- 
tion that the SSHA contributes unique valid 
variance to the prediction must be made with 
caution, 

The mean of the total male group of the 
present investigation was found to be 30.71, 
which would be equivalent to the goth percentile 
on the published norms based on 2,114 men from 
twelve different colleges, These means are sig- 
nificantly different at the one per cent level. 
This difference might be expected because many 
Students enroll in the How-to-Study courses 
because they have academic difficulty and poor 
study habits. A random sample of 50 students 
was drawn to determine the internal consistency 
of the survey for the present investigation, Hoyt's 
analysis of variance technique gave a reliability 
coefficient of .88 for this group. 


n Achievement (n Ach) 


The n Achievement test, as its name implies, 
is a projective device designed to measure one's 
need for achievement, A complete description of 
this instrument Ty be found in a book by 


. In many respects 
it is very similar to the Thematic Apperception 
Test (TAT) developed by Murray (16), but it 
differs from it in three major ways. First, al- 
though some of the TAT pictures may be used as 
stimuli, additional pictures have been specially 
prepared for the n Ach, Second, group testing 
is used. The pictures are flashed on а screen and 
the subjects write their stories, Third, the 
scoring system, though derived from some of 
Murray's concepts, is completely different from 
that used on the TAT, 

The primary assumption in this test is that a 
person with a need to achieve, а person who 
might be termed ambitious, driving, or highly 
motivated, will project this need in the stories 
he devises, The stories, then, can be objectively 
scored by counting the number of achievement- 
related cues. The more highly motivated a per- 
son is, the more achievement cues he is expected 
to project. 

Pictures A, B, D, and E (14, p. 375) were 
administered in that order. One Psychologist 
scored all stories according to the detailed in- 
structions (14, pp. 107-138). His scoring of 8o 
stories by 20 people correlated .8i with the 
scoring of these same stories by the authors 
(14, pp. 346-374). Furthermore, on another 


sample of 80 stories scored and rescored after an 
interval of a month, a score-rescore reliability of 
:91 Was obtained by this psychologist. 

А complete and detailed account of. the 
validity of the n Ach is given in The Achieve- 
ment Motive (14). No attempt is made here to 
reproduce this information. However, one study 
by McClelland (14) conducted on a sample of 
thirty Wesleyan College male veterans resulted 
in a correlation between п Ach and average 
grade of „51. When the results of a verbal and 
mathematical scholastic aptitude test were par- 
tialed out, the relationship between n Ach and 
Brades was reduced to 39 (which is still signifi- 
cant at the five per cent level of significance). 
Thus the predictive validity of n Ach has been 
at least partially established by its ability to 
predict grades. 


Opinion, Altitude, and Interest 
Survey (OAIS) 

The OAIS was developed by Benno Fricke (8) 
to measure certain nonintellectual personality 
factors important in academic success, In other 
words, it attempts to measure motivation. This 
survey consists of 396 statements presented in û 
paired fashion—a total of 198 items. 1 

The construction of the OAIS was base 
upon Science, Literature, and the Arts College 
male students at the University of Minnesota. 
Only students with ACE percentiles above 51 
were included in the criterion groups used и 
item analysis. Fricke reports about 70 per сЕ 
of SLA students score above the 54 percentile о 
entering University of Minnesota norms (8) 
р. 164). 

On the basis of ACE and honor-point таси 
(HPR) Fricke selected a group of students des 
ignated as achievers and another group pel 
ignated as nonachievers, He then performed а! 
item analysis to discover which items of the 
OAIS (scored configurally) differentiated Ex 
tween these groups. The next step was > 
establish the usefulness of these items ae aie 
cross-validation sample of incoming students. ГЕ 
combined high school rank (HSR), Ohio d a 
Psychological Examination, and the OAIS in a 
regression equation which revealed that the S 
Weight of the OATS Was significant at the ОП 
Per cent leve] of confidence. 

The zero order correlation coeficient (? 
between the ОД]$ scores and HPR for the 
total cross-validation sample of 209 шде" 
Was .40; the т for the 138 students having s 
Percentiles greater then 54 was .46; and the 
for the 71 students having ACE percentiles belo 
55 Was 28, ind 

Three limitations should be kept in min 
regarding the use of the OAIS in this study: 

1. The OAIS was validated on students with 


io 


| 
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above-average academic ability. The mean ACE 

(1947 form) raw score was 119, equivalent to the 
72nd percentile on norms based on entering 
freshmen in Minnesota colleges. Students in the 
Present investigation, on the average, scored 
lower. Their mean ACE (1952 form) raw score 
Was 100, equivalent to the 391 perctntile-n 
equivalent norms. 

. 2. The scoring keys were derived from an 
item analysis of male responses only. 

E The original purpose of the OAIS was to 
discover personality variables predictive of 
academic success prior to entrance to college. In 
the present study this instrument is being 
applied to students already enrolled in Moe 
Furthermore, these students have undergone а 
selection process in becoming members of the 
present study. 

Thus both the situation an 
the population have been altered in the present 
application of this instrument. 


d the nature of 


Survey of Opinions—Attitudes (SOO-A) 


The purpose of the SOO-A is to provide a 
quantitative measure of student attitudes toward 
class on a favorable-unfavorable continuum. It 
Consists of items 1-10 on the Survey of Opinions, 
а copy of which will be found in the Appendix. 
Each of these ten statements has five possible 
responses, The directions instruct the students 
to answer each question on a percentage basis. 
That i ‚ he is asked to estimate what percentage 
of the time he feels a certain way toward the 
class, 

This measure was devised by 


in this study simply on the basis © 
Validity of Zah Pao: Arbitrary weights were 
assigned to cach response to each question. ds 
Weight of four was first assigned to the response 
believed to be most favorable toward the class, 
three to the next most favorable, on down ex 
zero for the least favorable response чү sible 
item, Thus a student with the most favora 


atti ossibly get а 
attitude ў class could poss! ly 8 
toward the tudent with the 


Maximum score of forty, and a 5 could 
most unfavorable attitude toward the class 
Possibly obtain a minimum score of zero. e 

This questionnaire was administered ше 
times during the quarter to the students А is 
How-to-Study classes. On the basis of Es t 
Sponses of 145 students taking the EA en 
Ministration, a new series of weights was ass E eil 
to the responses using the method of He pes 
averages (15). The weights stabilized uet Н 
iterations with few changes in weights = of 
a reliability of 89. The third айю ниш nd 
the SOO.A is referred to as the ES н Аан 
Was given at the end of the quarter Wt! ўд 
lions to consider the entire quarter's p 


answering the questions. 


‚ the instructors 
f the face 


IN A HOW-TO-STUDY COURSE 15 


The Preferred Instructor Characteristics 
Scale (PICS) 


The primary purpose of this investigation is 
to discover the effect of different teaching meth- 
ods on various student outcomes. However, one 
of the important subsidiary problems is con- 
cerned with the effect of students' prejudice for 
or against a certain kind of teaching method on 
their outcomes under each condition. There was 
a need for some instrument to measure this 
prejudice, and the PICS was designed to meet 
this need. It is similar in design and theory to 
an instrument developed by N. L. Gage and 
others (9, pp- 17-19). 

A "cognitive-affective" continuum of instructor 
characteristics was postulated. A cognitive in- 
structor was defined as one concerned with the 
intellectual, abstract, subject-matter goals of 
teaching; the affective instructor was defined as 
being more concerned with emotional adjust- 
ments and student interactions in the classroom. 
The cognitive instructor corresponds to the in- 
structor-centered instructor of the present in- 
vestigation, while the affective instructor cor- 
responds to the student-centered type of in- 


structor. 
The instructors in this study proposed a 


number of statements that they believed char- 
acterized each of the two type of instructors, 
Then, to obtain some degree of face validity, 
they submitted the statements to three advanced 
graduate students in educational psychology and 
one instructor in humanities with directions 


to separate the items according to whether 
they were 


"affective" or "cognitive" as defined 


above. 3 | 
Ошу statements which were unanimously 
classified by all four judges, plus the two in- 
structors, were retained for the scale. Eight 
cognitive and eight affective items met this 
criterion. А j À 
Each cognitive statement was paired with 


affective statement to form a 64-item 
arison forced-choice instrument. This 
form was administered to two fall, 1954 How- 
to-Study sections to provide the basis for an 
item analysis. The item analysis revealed that 
two statements of each type were not discrimi- 
nating because they were either almost always 
chosen or almost never chosen. These four state- 
ments were discarded. The remaining six cogni- 

e items were again paired to 


tive and six affectiv г i 
form a 36-item paired-comparison forced-choice 
instrument. The order of each type of statement 


within each item was randomly determined, The 
resulting form appears in the Appendix, 

The PICS is scored in such a way that a high 
score indicates that the student prefers E 
cognitive-type approach, while a low score in- 
dicates preference for an affective approach. The 


each 
paired-comp 
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maximum score is 36, while the minimum score 
is zero. 

The reliability of the PICS was measured in 
several ways. It was administered to two different 
sections of the fall, 1954. How-to-Study class 
and item analyzed again. Thirty-four of the 36 
items yielded a phi coefficient of .20 or higher, 
indicating that the vast majority of items were 
successfully discriminating between the upper 
and lower 27 per cent of the distribution, 

A measure of the test-retest reliability was 
obtained by administering the PICS to 2i night- 
school students on two occasions with а four- 
week intervening period. A test-retest reliability 
coefficient of .88 was obtained, 

One further check on the internal consistency 
of the PICS was made by obtaining a random 
sample of fifty students from the present investi- 
gation. Using Hoyt's analysis of variance tech- 
nique (тт) а reliability coefficient: of фо was 
obtained, The PICS was administered on the 


first day of class to students involved in the 
present. study, 


American Council on Education 
Psychological Examination (. IGE) 


The ACE is widely known and used 
of scholastic aptitude, Because of thi 
discussion of its construction, 
scoring, reliability, and validity is unnecessary, 

The ACE scores for students in the present 
investigation were obtained from the files in the 
Student Counseling Bureau at the University of 
Minnesota. If a student had not taken the 1952 
form of the ACE, his score on one of the other 
forms was transformed to the 1952 raw score 
equivalent by means of conversion tables pro- 
vided by the Student Counseling Bureau. Of the 
111 randomly assigned students for whom ACE 
scores could be located, 58 had taken the 1952 
form, 42 had taken the 1947 form, and ıı had 
taken the 1937 form. 


as a test 
s fact a 
administration, 


Personal Data Card 


A personal data card was employed to collect 
information about cach student's college, cla: 
home and local address, telephone number, аве, 
and sex. In addition, the card used to record 
all the test data collected for cach student. 


Intercorrelation of Instruments 


Table ıı presents the intercorrelations of the 
evaluation instruments based on 80 randomly as- 
signed males on whom complete data were avail- 
able, " 

It should be noted that all measures which 
were administered twice had а test-retest Te 


from perfect. The two tests which purport, 10 
measure motivation (OAIS and n Ach) corre 
consistently positively with cach other, but le 
on the pretests does this reach significance att i: 
оз level. Only the final examination shows шу 
significant relationship with the ACE. The tw 
measures of achievement (SSHA and final ex: 
nation). correlate significantly but not 
with each other, 

It should also be noted that the GAR с, 
relates significantly with all measures of к c 
ment, while » Ach reaches significance al s 
05 level with only onc—the final Dc 
posttest, Liking for class (SOO-A-3) a 
have little relationship with any of the aues 
measures. The PICS is found to correlate he 
nificantly with the OAIS on both the pre- Hp 
posttest administrations, It also shows а ae 
prisingly high relationship with the SSA Pls 
test. Perhaps this is an indication that » 
PICS may have possibilities as a measure 
motivation, 


V. RESULTS 


p : 3 in 
The primary hypotheses as outlined 


TABLE gy 


INTERCORRELATIONS OF ТЕ 


хов SHA 


— SSHA Final 1 Si W XXE MAS " 

Variables ч ze Ina PICS — » Acl i Acl JAIS | OAIS ОО: 

i : (Pre) (Pos) (Pre) (Post) PICS (Pre) (Pon) (ir (os) S 
ACE -18 —.or 32 27% 0L еа E pr 
SSHA (Pre) svo i Су | кый, Ln 
SSHA (Post) INE Mn E po 20 
Final (Pre) -Sot 29 a3 e 2 
Final (Post) xb x p e 

CS ec 229 

ЕИ (Pre) .o6 -30t оо 
п Ach (Post) -4ot pe 04 
OAIS (Pre) AT +03 
OAIS (Post) zo 
SOO-A-3 


* Significantly different from zero at the .05 level, 
+ Significantly different from zero at the ‘or level, 
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Part II are tested first. Following this 
the subsidiary hypotheses are tested. As 
specified in Part II, two assumptions for 
the use of analysis of variance and covari- 
ance are tested. in these analyses. The 
variance within each cell is assumed to be 
homogeneous, and for all covariance 
problems the linear regression coefficients 
within each cell are assumed to be homo- 
geneous, In the text of this part, no fur- 
ther mention is made of these assump- 
tions unless they appear to be violated 
in a particular analysis. Each analysis in 
the tables of results is marked if either 
assumption is violated. 

For each problem in the investigation 
variance has been cal- 


an analysis ol 
scores and on the 


culated on the pretest 
posttest scores. In addit 
analysis has been calculatec 

test scores adjusted [or pretest scores. The 
analysis of most concern is the covariance 
analysis because any differences in post 
lest scores are adjusted for differences 
that might have existed in the initial 
Scores. 


ion a covariance 
lon the post 


Tests of Primary Hypotheses 
The primary null hypotheses of the 
present investigation may pe stated as fol- 
lows: 
among mean scores 


1. No differences exist 
attributable to 


9t the criterion instruments 
different methods of teaching- 

з. No differences exist between meat pom 
of the criterion instruments attributable to € er- 
ent time-instructors. | 
. 3. No interaction effect exist 
Ing method and time-instructor. 


1 hypothese 


s between teach- 


s аге tested 


The three nul 
instruments 


for each of the four criterion 
plus the SOO-A-3. For each instrument а 
Separate analysis is made 
females combined and for ™ 

No analysis of females alone is ma E 
cause of the extremely small frequen 
Within each cell. On the SSHA no ana- 


for males and 


r males alone. 
e be- 
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ysis of males and females combined is 
made because of the different scoring keys 
for each sex. 

Each analysis of variance (AV) and 
each analysis of covariance (AC) is sum- 
marized in one row of Table 12. The 
table indicates whether the analysis was 
performed on pretest scores (pre) or post- 
test scores (post), and whether males and 
females combined (MF) or males only 
(M) were analyzed. 

"Table 12 reveals that the primary null 
hypotheses on posttest scores are all ac- 
сергей on the final examination, the 
SSHA, and OAIS. However, the analysis 
ariance of п Ach posttest scores for 
and females combined reveals that 
the hypothesis of no method difference 
has been rejected at the .o5 level. 

What is the nature of the differences 
between methods? Which method pro- 
duced the highest level of motivation and 
which the lowest? The mean scores for 
each method are presented in Table 13. 

The adjusted mean score in Table 13 
is the posttest mean score adjusted for the 
pretest mean score by means of the re- 
gression coefficient. It is the difference in 
adjusted mean scores that is of impor- 
tance. The students taught by the eclectic 
method scored highest. The students 
from the instructor-centered sections 
were next highest, and the students in 
udent-centered sections scored low- 


of соу 
males 


the st 
est. The chances are less than one in 
twenty that differences as great as these 


arisen by chance alone. 
However, what is the significance of 
the difference of each method compared 
with each other? Scheffé (18) has sug- 
gested а technique for judging contrasts 
in analysis of variance. In judging con- 
trasts in analysis of covariance the pro- 
cedure is virtually identical except that 
adjusted mean scores are contrasted and 
e estimated variance of the difference 


could have 


th 
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TABLE 12 
SUMMARY or F VALUES TESTING 


PRIMARY HYPOTHESES 


F Values for each Sourc 


е. 
а f Variation 
Analysis of E —$— مج ج چ ج‎ 
Total Pretest Jariance Mor | Ведае 
Variable df or or Co МЕ | jetween 
Posttest OF ( з | Among Time Inter- 
Matianee | Methods Instruc- action 
tors pe 
Final Examination тїї Рге AV MF <1 <1 2.77 
a тїї Post AV MF 1.02 <1 1.82 
110 Bost E МЕ | <1 1.55 =) 
2 re А | 44% 2.22 2.15 
82 Post AV M <1 <1 3-05 
81 Розї АС м ©з «1 dr 53__ 
55НА 82 Рге AV M 2.28 <1 pe. 
82 Post AV M 1.58 1.71 <r 
81 Post AC M <1 1.55 = za 
n Ach 109 pie AY МЕ «1 2.26 $ : 
109 ost F 3.72° <1 
108 Post AC MF 3 p «1 = 
80 Pre AV M 1.45 <1 У-У 
80 Post AV M X 84. «1 
79 Post AC M «1 <1 sa ж 
k| 
OAIS 109 Pre AV MF 2.60 «1 =1 
19 Post AV» MF 1.58 <1 2x 
я ‘ost AC MF <I <1 p 
Bom Ao M سوا‎ $i d 
о: °ч 1 1.6 
79 Post ACh M © i =: = 
SOO-A-3 109 Post AV MF ш pony "6.034 
о Post AV M 2 ai ЕСА 


2 Assumption of homogeneous v. 


re Н rise ariances violated, = Е 
Ssumption of homogeneous regression Coefficients vi 

* Significant at the +05 level. iolated. 

4 Significant at the „от level. 


between adjusted means i 
creased (5, p. 79). 

A null hypothesis of no di 
mean n Ach scores for е 


s slightly in- 


fference in 
ach pair of 


centered teachin 


in- 
B method tends to be 1 
termedj 


h- 
ate between the other two met! 


i 4, : TABLE 13 
methods is tested. Scheffé's technique re „ ACH MEAN Scores* АззостАтЕр wrt EACH 
veals that only the contrast between the ETHOD 
eclectic and student- 


centered methods 
five per cent level. 
lo not reach signifi- 


is significant at the 
The other contrasts d 


(Males and Females) 


Method 


cance. | Test Togtructor- gem Students 

Thus we may conclude, with less than Was der 
one chance in twenty of being wrong, Pretest : "T 
that for this population the eclectic — Posttest xi. qn 13-9 15:03 
teaching method motivates students toa Gusted ti à p 
higher level than does the Studenten. ——— — E 17.5 


tered teaching method. The instructor- 


| i а 
to bi et “cores include a constant of то adde 
€ach score to eliminate negative numbers. 


————— M 1 
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ods, but contrasts involving the instruc- 
tor-centered method do not reach signifi- 
cance. 

It is interesting to note that all three 
null hypotheses are accepted when males 
only are analyzed on the л Ach. Appar- 
ently, then, the majority of the contri- 
bution toward method differences is ас 
counted for by the females. 

The present results tend to raise more 
questions than they answer. In previous 
research. (14) the different motivating 
conditions were established by means of 
relatively brief directions. Males were 
affected by differences in these brief moti- 
vating conditions in the expected direc- 
tion, but females were not. Females 
seemed to respond in the same manner 
whether they were motivated under a re- 
laxed condition or under an achieve- 
ment-oriented condition. The present in- 
vestigation, by extending the stimulating 
conditions over a ten-week period of 
time, may have been long enough to dif- 
ferentially motivate the females. In addi- 
tion, the social climate may have been 
sufficiently different under each method 
to cause different perceptions and TERS 
tions. Other explanations ате possible. 
Further research is necessary tO isolate 


the exact nature and etiology of the sex 


differences associated with 7 Ach. 
action 15 evident for 


A significant inter 
the ais of variance of the SOO-A-3. 
This interaction precludes any test of the 
main effects because of the assumption of 
No interaction in the model used with 
Unequal frequencies in cells (4)- Я 

The SOO-A measures attitudes towar 
Class оп a favorable-unfavorable con- 
tinuum, The third administration of the 
SOO.A is analyzed because it was СОП- 
ducted at the end of all instruction when 
Students were asked to view the entire 
quarter’s work in answering the ques 


tionnaire. 
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TABLE 14 


MEAN SOO-A-3 SCORES 
(Males and Females) 


mn Method 
Instruc- Total 
Instructor- .. Student- 
rep Centered Eclectic Centered 
A | 28.1 25.8 29.8 28.0 
32.8 27.5 25:1 28.6 
Total | 30.3 26.7 27.9 28.3 


The reason for the significant interac- 
tion can be seen in Table 14. A higher 
score tends to indicate a more favorable 
attitude toward class. The interaction is 
due to the fact that the instructor-cen- 
tered section of time-instructor B scored 
higher than the corresponding section 
of time-instructor A, but the student- 
centered section of time-instructor A 
scored higher than the corresponding 
section of time-instructor B. The differ- 
ences are not as wide in the eclectic sec- 
tions. It appears that each instructor (or 

ossibly the time of day) had a method 
with which he could best please the stu- 
dents, but the same method did not 
please each group of students equally 
well for each instructor. 


Tests of Subsidiary Hypotheses 
on Preference Levels 


One of the subsidiary problems of this 
investigation involved the possibility that 
a preconceived bias concerning the teach- 
ing methods might influence the out- 
ssociated with each method. The 
d in Part IV, was designed 
to measure this bias. On the basis of PICS 
scores the random students were classified 
categories of approximately 
equal size. The high scoring third are 
those students who tend to prefer a more 
directive ог instructor-centered approach; 
the low scoring third would tend to pre- 


fer the student-centered method; while 
those in t 


comes а 
PICS, describe 


into three 


he middle might be likely to 
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prefer some combination of both, pre- 
sumably the eclectic method. 

The exact design for this analysis is 
set out in Part П. In brief, the purpose 
of this analysis is to see if differences on 
the criterion instruments appear to be 
related to preferences for the different 
methods as measured by the PICS. At the 
same time it is possible to discover any 
interaction effect between preference for 
instruction and the actual instruction 
given. A covariance analysis adjusting 
posttest scores for pretest scores has been 
carried out to test the following null hy- 
potheses: 


1. No differences exist among mean scores of 
the criterion instruments attributable to differ- 
ent teaching methods, 


2. No differences exist among mcan scores of 
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the criterion instruments attributable to differ- 
ent PICS levels. 

3. No interaction effect exists between method 
and PICS level. 

The first hypothesis is identical to the 
first of the primary hypotheses, and con- 
sequently the results in the following 
analysis are identical. It is necessary to in- 
clude this hypothesis again in order to 
isolate the variation due to method dif- 
ferences and provide a means for measur- 
ing the interaction effect between actual 
teaching methods and preference fou 
method. The results are summarized in 
Table 15, which utilizes the same sym- 
bols as Table 12, 

A highly significant F value can be 
noted for the covariance analysis of 
SSHA scores. Apparently then, study 


TABLE 15 


SUMMARY F VALUE 


ESTING SUBSIDIAR 


ү Hyroruk 


ON PREFERENCE LEVELS 


Analysis or F Values for Each Source 
М Pretest alysis ol f Variation 
Variable ra i Ps Variance Mor —— е 
if Posttest OF Co- MF | Among Among Inter- 
РВЕ | Methods PICS action 
UN CVC ج‎ 

Final Examination тїт Рге AV MF E ear m 
un Post AV МЕ S E 1.76 

о Боз АС МЕ <1 at 1:75 

8 ге AV M | 1.58 
Sr Post AV M 2 | Ws <1 
2 2 2 | = ® n 

| pum 
SSHA a m ay кир  ———Z 
22 Post AV M |? a Е k^ ёл 
ost АЯ «3t 
» AC M | <i 7«30* E = 

д 0 оо у мр г 
E Post AV МЕ 3.86 21 1.44 

ч Post AC MF 3. ib E 1.70 
До Pre AV M : E ES <1 

"B Dost AV M ч es 1.10 

79 Post AC M E gs 1.58 

OAIS 109 Pre AV u———— $T ےھ‎ 
тоо Розї AV МЕ 131 soa a 
108 Post ACa МЕ +43 2.48 к 

80 Pre AV M <r E: 21 

о Post AV M e d = 59, «1 

70 Post ACa M Be 1 pe: 21 


a Assumption of homogeneous re; 
» Significant at the .os level. 
e Significant at the .or level. 


gression coefficients Violated 


R 
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TABLE 16 
zAN SSHA PRE- AND POSTTEST SCORES FOR 
Нісн, MEDI Low PICS GROUPS 
(Males) 


PICS Levels 


Administration 


Low Medium High 
29.76 29.82 33.08 
| 25.07 29.21 37.38 


habit changes do show some relationship 
to the students’ original preferences fora 
certain type of instruction; and since the 
interaction is not significant, it can be 
concluded that this is true regardless of 
the type of method they actually receive. 
The nature of this relationship can be 
seen in Table 16. 

Since the differences 
and the interaction are not significant, 
the scores from all methods have been 
pooled to form the mean scores of Table 
16. The trend was for the low PICS 
group to decrease their SSHA scores, for 
ate themselves 


among methods 


the medium group to r 
about the same, and for the high group 
to increase their SSHA scores over the 
period of instruction. It appears that the 
male students who originally preferred 
а more cognitive or directive teaching 
method showed the most improvement 
in their self-ratings On study habits. 
Those who originally preferred a more 
affective or student-centered approach 
tended to rate their own study habits 


lower after instruction than they did be- 
fore no matter what type of instruction 
they actually received. 


The reasons for this result are not appare 
However, one hypothesis таў $ 
gj Ната the student W 
ng out of the how-to-stucy 
pleat who just wants to pick up two 
us the other hand, since the E : 
suns El correlated with сас 
shi may simply be an artifact 0 
MP, and both tests are simply me 
Positive academic attitude—an attitude : 
Acterizes students who are academically succes 


asures 0! 
that char- 
ful. 
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However, this does not explain why the PICS 
and the posttest SSHA correlate „у while the 
PICS and the pretest SSHA correlate only .12. 
It is conceivable that the PICS is a measure of 
motivation toward academic success, but further 
research is needed to substantiate this hypothesis. 

The analysis of variance of OAIS posttest 
scores for males only does reach significance at 
the .05 level. The covariance analysis (adjust- 
ing for initial pretest differences) is questionable 
because the assumption of homogencous regres- 
sion coefficients has been violated. However, it 
seems questionable to accept the conclusions of 
the analysis of variance of unadjusted posttest 
scores. Differences among PICS levels also appear 
in the pretest scores, although not quite reaching 
significance (Е = 2.59). An exact test is not 
ble, but it appears that differences among 
levels on posttest scores are largely the 
result of initial differences. Furthermore, the 
initial differences are possibly due to the cor- 
relation of the PICS and the OAIS аз noted in 


Part IV. 
Tests of Subsidiary Hypotheses on 
Ability Levels 


bsidiary problem con- 
cerns the scholastic ability level of stu- 
dents and its effect on outcomes under 


each teaching method. The possibility 


that bright students might have different 


outcomes than those less bright, and the 


possibility that there might be some in- 
een ability level and the 


actual teaching method used are to be 
tested here. The specific hypotheses to 


be tested may be stated as follows: 

1. No differences exist among mean scores of 
the criterion instruments attributable to different 
teaching methods. . 

2. No differences exist 
the criterion instruments 
ent ability levels. 

з. No interaction 
ing method and abil 


The second su 


teraction betw 


among mean scores of 
attributable to differ- 


ion effect exists between teach- 
ity level. 


Here again the first hypothesis repeats 
the first of the primary hypotheses in or- 
der to isolate method differences and to 
provide for the test of interactions. The 

immarized in Table 17. 


results are SU 
Examination of Table 17 reveals no 


significant F values for posttest scores 
ot already been mentioned. 


that have n 


1o 
Г 
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TABLE 17 


SUMMARY OF F VALUES TESTING SUBSID 


IARY HYPOTHESES ON ABILITY LEVELS 


ЕЗ? Е Е Values for Each Source 
Бара Analysis of of Variation 
retes Jariance c а 
Variable з ог ог Со- M F Among 
Posttest variance Among ACE Inter- 
Methods — Levels action 
Final Examination тїт Рге АУ МЕ <1 1.53 s3 
III Post AV MF 1.05 2.73 si 
IIO Post AC MF «1 2.11 <1 
81 Pre AV M 1.37 3.40" = 
81 Post AV M <1 2.30 bo 
8o Post AC M <r <1 20 

55НА 81 Pre AV M 2.59 <1 e 
81 Post AV M 1.13 XT т 

Зо Post AC M zi 1.17 EA 
n Ach 108 рге AV MF <1 2.39 E: 
108 Post AV MF 4.04 <1 1.43 
107 Post AC MF 3.51" «1 1:97 
79 Pre AV M 1.80 1.77 S 
79 Post AV M 1:66 <1 s 
78 Post AC M <1 <1 49 
OAIS 108 Pre AV MF — <1 p 
108 Post AV MF 1.45 1.32 Тино 
107 Post AC MF <i “з pos 
79 Pre AV M 4.88" <I Eos 
79 Post AV M 1.60 <I ES 
78 Post AC M <r «1 140% 


a Assumption of homogeneous variances violated. 
^ Significant at the -05 level. 


Apparently, scholastic aptitude 
ured by the ACE bears no signi 
lationship to outcomes on the 
used in this study, There does n 
to be any tendency for a brigl 
to be more successful under 
ing method than another. 


аз meas- 
ficant re- 
variables 
Ot appear 
1t student 
one teach- 


VI. SUMMARY AND CONCLUSIONS 


The primary problem in the present 
investigation is to discover any achieve. 
ment or motivational differences that 
may result from three different methods 
of teaching in a how-to-study course. The 
population consists of al] University of 
Minnesota students who elected to take 
Personal Orientation 1, Ноу to Study, 
in the winter quarter of 1955. Approxi- 
mately 63 per cent of the 192 students 


enrolled were randomly assigned to em 
teaching methods taught by two instru 4 
tors at two different times of the day- ^ : 
though nonrandom students do not e 
pear to be essentially different from t r- 
random students on any of the digitis 
istics examined, the analysis of results 
confined to the random students. 
The three teaching methods 
termed instructor-centered, student- - 
tered, and eclectic. The instructor-ce" 
tered method emphasizes the intel t 
content of the course and consists Р? 
marily of lectures and instructor-direct® " 
activity. The student-centered арр d 
‘ends to emphasize the more affective “ 
pects of the classroom and deals ies 
student problems by means of commit he 
Work and Studentled discussions. i 


are 
cen 


LIL—————r‏ چ 
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eclectic method consists of a combina- 
tion of the previous emphases and pro- 
ceeds primarily by means of instructor- 
led class discussion interspersed with a 
variety of other techniques. Checks on 
the consistency of instruction by each in- 
structor in each method by means of 
coordinated planning, a student rating 
оп instructor behavior, and judges’ rat- 
ings of class recordings tended to affirm 
the consistency of instruction. 

A design embodying the principles of 
replication, randomization, and control 
is utilized. Null hypotheses are con- 
structed to test the relationship of 
method, time-instructor, and interaction 
to pretest and posttest measures of mo- 
tivation and achievement. Subsidiary 
analyses of the effect of original prefer- 
ence for instructional method and ability- 
level on motivational and achievement 
Outcomes associated with each method are 
conducted. Analysis of variance and 
Covariance by least-squares analysis is the 
Primary statistical tool. 

_Five criterion instruments are utilized. 
The final examination is a gg-item, ob- 
Jectively scored, test of knowledge about 
the course content. The Survey of Study 
Habits and. Attitudes (SSHA) was used 
to obtain student self-ratings of study 
habits and attitudes. The n Achievement 
test (n Ach) is a projective technique de- 


51 Н i 
— to measure achievement motiva 
Sons ре Opinion Attitude and poste 

ге! 


est Survey (OAIS) is a configurally sco! 
ue моц: which has been shown to or 
ute significantly to the prediction o 
wn Point ratio. All four of these ^ 
Bhd n ts were administered both uc 
fifth after the instructional peor. T 
eae is the Survey of pin 
deg: Htudes (SOO-A), a questionnz 
o, ned to measure student attitudes 
Ee class on a favorable-unfavorable 
Шиит. 


The results of this study may be stated 
as follows: 

1. Significantly different motivational 
outcomes as measured by the п Ach are 
found among the three teaching methods 
for males and females. Students in the 
eclectic section were most highly moti- 
vated as measured by n Ach; the in- 
structor-centered students were second; 
and the student-centered students showed 
the least increase. These method differ- 
ences disappear when males only are an- 
alyzed, indicating the major contribution 
of females to these differences. 

2. Differences in outcomes as measured 
by the final examination, the SSHA, and 
the OAIS are not significantly different 
for the three methods. 

3. On the final administration of the 
SOO-A for males and females an interac- 
tion of method with time-instructor is 
found. The method which was most suc- 
cessful in pleasing the students under one 
instructor (or possibly at one time of 
day) was not most successful under 
another instructor (or at another time of 
day). А 

4. When students аге categorized on 
the basis of their original preference for 
teaching method, it is found that stu- 
dents who originally expressed a prefer- 


ence for a more cognitive-type instruction 


increased their self-ratings of study habits 
апа attitudes (SSHA). Students who 
originally preferred a more affective or 
student-centered type of instruction 
tended to lower their self-ratings. This 
was true regardless of the type of instruc- 
tion they actually received. On the other 
variables no such differences were ob- 


served. 


g. When students are categorized on 
5. 


the basis of their scholastic-ability level 


measured by the ACE, no significant 
«а г achievement outcomes 


ivational 0 x 
et ation to ability level. Fur- 


are found in rel 


°4 


thermore, there is no tendency for bright 
students to have any different outcomes 
under one teaching method than 
another. 

Previous research on teaching methods 
has dealt with extreme teaching methods. 
This may have been done to maximize 
the possibility of obtaining significant 
differences. The present investigation il- 
lustrates the importance of considering 
an intermediate eclectic method combin- 
ing the advantages of both the typical 
lecture and the typical group approach, 
Many of the studies on teaching methods 
cited by Farquhar (6) could be profitably 
repeated with the introduction of 
eclectic method. It could well be th 
combination method is far 
either extreme 


an 
ata 
superior to 
in implementing many 
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educational objectives. Certainly the re- 
sults of this study point in that direction. 


In designing an investigation similar to о 
present one several improvements could. be UA 
Many students were lost to the analysis beca E 
the randomization procedure was ieam ا‎ 
it were possible administratively to assign ss 
students randomly, the precision of the Bc a 
ment would be increased, Another айин 
problem concerns the use of a noui 
trol group. No evidence can be obtained by 
the present design as to whether instruction А, 
any method is more effective than по instante 
at all. Where possible, future research ЧЕ 
this nature could be improved by more comp i 
randomization and the institution of a А 
ment control group. Complete. details of md 
investigation, including reviews of literature 4 4). 
the raw data, may be found in two sources (б, 13 

Further research in other subject matter Eie 
and with other populations is necess ey e tudy 
Stantiate the results of this study. The 8 an. 
poses many questions which can only be # 
swered by controlled experimentation. 


APPENDIX 


PREFERRED INSTRUCTOR CHAR 


Directions: 


What kind of an instructor do you prefer? In 
the following items you will find two instructor 
characteristics paired. From each pair choose the 
one characteristic you most prefer, Then mark 
your choice in the Proper column on the special 
answer sheet. Do not omit any items, This is to 
find out your preferences, There are no right 
or wrong answers, 

I prefer an instructor who: 
1. а. is an expert, 
b. treats us as mature people, 
‚ а. makes the classroom pleasant, 
b. thinks logically, 
. understands our point of view, 
b. is well known in his field, 
. is dedicated to his students. 
b. is dedicated to his subject, 
5. @ thinks logically, 
b. is friendly, 
. is well known in his field. 
b. makes the classroom pleasant, 
. is interested in us. 
b. covers all the materia], 
а. is dedicated to his students 
b. knows the theoretical back 
subject. 
. thinks logically. 

b. treats us as mature people, 

10. a. is friendly. 


e 


ون 


Bround of his 


ACTERISTICS SCALE 


. is well known in his field. 

. Covers all the material. 

b. understands our point of view. 
. is interested in us. 

b. is dedicated to his subject. 
«dsan expert, 

-is dedicated to his students. 


14. а. is well known in his field. 
b. treats us as mature people. 
15. a 


. Covers all the material, 
b. makes the cl 
. underst 


assroom plcasa 

ands our point of view. 

. is dedicated to his subject. 

. is interested in из. 

b. knows the theoretical background 0 
Subject, 

.ds friendly, 

+ covers all the material. 

+ makes the classroom pleasant. 

b. is dedicated to his subject. 


- knows the theoretical backgrount 
Subject, 


[ his 


1 of his 


understands our point of view. 
: IS Interested in us, 
. isan expert, 
* @ is dedicated to his students. 
b. thinks logically, 
+ treats us as mature people. 
b. Covers all the material, 
> 15 dedicated to his subject. 
.ds friendly, 


эб. 


30. 
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. a. makes the classroom pleasant. 


b. knows the theoretical. background of his 
subject. 

а. js an expert. 

b. understands our point of view. 


. а. is dedicated to his students. 


b. is well known in his field. 


га, is dedicated to his subject. 


b. treats us as mature people. 


ба. ds friendly, 


b. knows the theoretical background of his 


subject. 
а. js an expert. 
b. makes the classroom pleasant. 


HOW-TO-STUDY COURSE 


gı. a. thinks logically. 
b. is interested in us. 
. treats us as mature people. 
b. knows the theoretical background of his 
subject. 
. is an expert. 
b. is friendly. 
. thinks logically. 
b. understands our point of view. 
35. a. is interested in us. 
b. is well known in his field. 
36. a. is dedicated to his students. 
b. covers all the material, 
Check to see if you left any blanks. 


REFERENCES 


- BIRD, C., & Віко, Donor М. Learning more 


by effective study. New York: D. Appleton- 


Century, 19 


E, W. The teaching 
of research since 
52, 51-68. 

, W. H. Survey 
New York: 


of psychology: a survey 
1942. Psychol, Bull, 1 
W. Е. & HOLTZMAN 
of study habits and attitudes. 
Psychological Corp. 1953 
IMAN, R, The lea 
way factorial experiment 
frequencies in the cells. 
paper, Bureau of Educationa 
Univer, of Minnesota 


wes analysis of à 
with unequal 
Mimeographed 
| Research, 


GERTRUDE - 
mental designs. New York: Wiley, ! 
igation of tl 
teaching methods 


to study course. 
of Min- 


pe 


lationship of three р 
student behavior in а how 
Unpublished Ph.D. thesis, Univer. 
nesota, 10 


sith groups. Min- 


(Accepted for Р 


No. 406). 
10. GOULDEN, C. Н. Methods of statistical analy- 


sis. New York: Wiley, 1939. 
. Hovr, С. J. Test reliabili 


estimated by 


ı1 J 
analysis of variance. Psychometrika, 1941, 
6, -160. 

12. JOHNSON, p. O. Statistical methods in research. 


New York: Prentice-Hall, 1949. 

j. D. An investigation of the 
effect of three teaching methods on motiva- 
tional outcomes in a how to study course. 
Unpublished Ph.D. thesis, Univer. of Min- 


nesota, 19 


13. KRUMBOLIZ, 


D, p. C., ATKINSON, J. W.. CLARK, 


ii R. А Е. 1. The achievement 
motive. New York: Appleton-Century- 
Crofts, 1953- 

15. MITZEL, н. E., & Hoyt, C. J.A methodologi- 


reciprocal averages technique 
a attitude scale. J. counsel. 
т, 256-259. 

( Explorations in per- 
York: Oxford Univer. Press, 


cal study of 
applied to ar 
Psychol. 

16. MURRAY, H. 
sonality. New 


ncapolis: Burg Publishing Co., 1954 1938. 
иске, B, С. The development of an em ч вош: son, Ё. Р. Effective study. New York: 
pi cally validated personality test employ- 17. т. 04б: Ў 
ing configural analysis for the md 18 pf, Н. A method for judging all con- 
of academic achievement. URE tasts in the analysis of variance. Bio- 
Ph.D. the ey, of Minnesota, 1051 ika, 1983» 40, 87—104. 
СА E е [инее $ & STONE, G. ne Ў О йен: of Minnesota Col- 
"AGE, N. L., LEAVITT, G.S, & > ir pupils 19 Bulletin of the БИ 010, 
Teachers’ understanding e ШЕ рае " lege of Science, Literature and Arts, 1953+ 
and pupils ratings 9 ther Ял. 56, No. 23- 
Psychol. тп 1955, 69, No. 21 (W Ва - 
ublication March 7. 1957) 


26 JOHN D. KRUMBOLTZ AND WILLIAM W., FARQUHAR 


SURVEY OF OPINIONS 


Е Е is course. 
s — AGFSR 5.1 am glad I took this 
Directions: А ine AGESR 6.1 think class time is well spent 
The purpose of this survey is to determine AGFSR 7.1 look forward to cach clas 
your opinions about this course. Please be frank meeting. 
and honest in your answers. Naturally, this will AGFSR 8. This clas is interesting to me. 
not affect your grade in any way. It is important AGFSR 9.1 feel the instructor wants me 
for the instructor to know how the class feels — ^ to;do rell. 
em. each oF шее. questions, AGFSR 10, І feel this is an important class 
To help you in answering, the following terms < тше. 
ы been defined on a percentage basis as fol- BESIR de Phe orucior encourages us 
ows: ixi үп questions. 
to answer our own q 
A—Almost always—from 86 to 100 per cent of AGFSR iz. The problems. we work w a 
the time arise out of the questions 
G—Generally—from 66 to 85 per cent of the suggest. 2 m. 
пше AGFSR 13. We have opportunities to rode 
F—Frequently—from 36 to 65 per cent of the pare our ideas and met 
time i "si lass. 
with others in the c = 
= imes— > ivities 
5: ELE from 16 to 85 per cent of the AGFSR 14. We evaluate our own activi 
ime h 
J ; in class, 
R-—Rarely—from о to 15 per cent of the time 
у: 5P 


he 
AGFSR 15. We work with the rest of t 
Circle the letter corresponding to your own 


class to get answers to our que 
opinion: tions. 
AGFSR LI enjoy this class, AGFSR 16. The instructor lectures. hat 
AGFSR 2. І feel this class has been valu- АСЕ SR 17. The instructor determines W 
able to me, topics will be discussed. jea? 
AGFSR 5I feel “at home” in this class. АС FSR 18. The assignments are very С 
AGFSR 


4. This class has been well taught. and definite. 


SCORING KEYS 


Preferred Instructor Characteristics Scale Survey of Opinions 
(PICS) 


i . І бла! 
Тһе following weights, derived from the 


A plus one is given for each of the following iteration, were used: 


cognitive responses: 


SOO-A 500-В 
па 7b aga 19.b 25.0 gia 
2b Bb 14-а 20.a 26.4 32. b Item Response Item Response 
bE ġa 15.a 21.b 27,b 33. a AGFSR AG FSR 
4-0 10.b 16.b 22.b 98 а 34. a 1 1344 
5a ia 17.0 es5.b 29. b. 35. b d. : i ^ а Ё i к ° 3 4 4 
б.а 12.0 18.b 2да 30. а 36. b E HESE үе 13. 0 2 2 3 
4 43200 14. o 2 2 3 4 
5 432210 i. e 32 
9$. 43 2 a б 66. BY € 
T. 4 4 $a Ms ай у 
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INTRODUCTION 


O NE of the most obvious objectives of 
counseling is to increase the coun- 
selee’s knowledge of himself. This objec- 
tive is justified on the grounds that a 
client must know himself if he is to make 
satisfactory adjustments to the difficulties 
which brought him to counseling. This 
justification is particularly pertinent 
when the problems of educational and 
vocational planning are considered. A 
client must have a reasonably accurate 
estimate of his mental ability before he 
can intelligently decide whether to enter 
college or to pursue a professional career. 
Likewise, he needs some knowledge of his 
interests before he can logically choose 
a curriculum or a vocation in harmony 
with them. The word "before" has been 
italicized in the preceding sentences to 
highlight a significant point in connec- 
tion with this investigation which studies 
a criterion of counseling effectiveness. 
The criterion compares one perception of 
self with an external measure of self, 
and notes agreement prior to counseling. 
À second comparison of self-perception 
and external measure is made after coun- 
—_ 


Ра ‘This study was aided by a 8 
"arch Fund of the Departmen i 
niversity of California. The cooperation of 
Obert Brownlee, Principal, Demonstration S. 
Ondary School, made it possible to «а i 
Study in his school. I. Aileen Poole assisted a 
the collection of the data. E. Wayne V D dol 
oseph Martin helped in the analysis of the Ea 
With the use of the Computer Laboratory 9 
hiversity, 


nt from the Re- 
t of Education, 


seling, and agreement again is noted. The 
criterion is the change in agreement be- 
tween the first and second comparison. 
The criterion studied combines good 
features of both sociological and psycho- 
logical criteria. The sociological aspects of 
the criterion are, of course, found in the 
postcounseling behavior of the client. He 
cannot score favorably on the criterion, 
except by accident, unless he has in- 
creased self-knowledge. Hence, an im- 
provement in such knowledge creates a 
“before” situation, subsequent to which 
the client is more able to show improved - 
behavior. 

The criterion in this investigation 
might be viewed also as a psychological 
one. It differs, however, from the Q sort 
and similar criteria used by Rogers and 
others in that it is based on an objective 
measure, a test score. Rogers (8) simply 
compared one perception of self with an- 
other perception of self in order to note 
changes. The present study compares 
self-perceptions obtained in the form of 
sellratings with objective test data. 


Terms Used in this Study 

For the sake of brevity, certain terms 
will be used to denote concepts which 
would require fuller explanation if the 
following definitions were not made. The 
terms “congruence” and “agreement” are 
used interchangeably. They are used to 
indicate coincidence between two or 
more measures. “Score” represents the re- 
sulting number when the indiyidual's 
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actual test score is converted (in the man- 
ner to be described) to its corresponding 
step on a five-point scale. "Score" means, 
then, the quintile rank-equivalent of the 
test score. The word "rating" is used to 
indicate the numerical step on a five- 
point scale which the individual checked 
when asked to rate himself. 


DESIGN or STUDY AND COLLE 
ОЕ DATA 


‘TION 


The purpose of this study was to in- 
vestigate a variable which appears to pos- 
sess the quality of being a sociological- 
psychological criterion. To accomplish 
this purpose, specific questions were 
formulated to be answered by the data 
collected in the experimental situation. 

1. The first of these questions was: 7s 
there a significant difference between the 
agreement of precounseling rating with 
score and the agreement of postcounseling 
rating with score? A related question was 
also asked. Are the counselees’ changes in 
agreement in the direction of greater 
agreement? In other words, do the com- 
parisons made before counseling differ 
significantly from those made after coun. 
seling; and does counseling increase the 
congruence of self-ratings and scores? It 
was anticipated that if these questions 
were answered in the affirmative, the fol- 
lowing one could be asked of the data. 

2. Does the counseled group at the 
conclusion of the experiment show 
ratings which are congruent with 
than the noncounseled group? This 
tion formalized the €xpected differences 
between counseled and noncounseled 

groups of students. In contrast, the first 
question made explicit the idea th. 
number of agreements would change 
from before counseling to after counse]- 
ing. In this case, each counselee acted as 
his own control. In the other, compari- 
sons were made between groups of stu- 


more 
scores 
ques- 


at the 


i i сіра іп 
dents unlike in their participation 1 
counseling. 


Locale and Population of the Study 


The basic data for this study were aA 
during the summer of 1953 at the ied iol 
tion Secondary School operated by the og 
sity of California in cooperation Nit p er 
land, California, public schools. The sum * 
high school completed its sixteenth Selon 51 
1953, at which time the total enrollment was PN 
students, Sixty-nine per cent of the students er 
residents of Oakland, go per cent came Ir ү 
other California communities, and 1 per pes 
from outside of California, The students ias 
from 58 high schools. Their median age proi 
years, 11 months, and their median grade p pM 
ment during the preceding semester was the 
ond semester of grade 10. H 

A poll of the students’ reasons for aliens? 
the summer school indicates that 27 per Ee 
were there to repeat a subject or to Даре. 
the mark, 25 per cent to gratify special TE 
and the remainder were completing теч s 
for college entrance ог high school Кеш 
In comparison with national norms, the er gu 
population was skewed in the direction S 
perior ability on a test of mental ability. - 

All students who during the preceding ih 
mester had been enrolled in grades nine p pen 
twelve were required to take a battery 9 This 
during their so-called “guidance period.” 1 no 
Period was regularly scheduled but paca têd, 
credit. The students, after testing Was comp a 
Were not expected to be at school during tl 
guidance period, ex- 

The purpose of the testing program ipe 
plained to the students at the first testing SCR in 
They were told that they were ракі раба т, 
ап experimental testing program which A sity 
ducted for research purposes by the Ls the 
of California, They were informed also tha they 
test results would be available to them if was 
wished to talk with a counselor. No ig the 
made to induce students to make use О Jents 
counselors. At the first testing session, stud talk 
were asked whether or not they wanted to, cate 
with a counselor, The data in Table 1 иш 
the number who wished to be counseled as The 
as the number Who were actually counseled. ents 
discrepancy between the total of 0951 stuc 1 is 
enrolled in the School and the 485 in I eit 
accounted for by a variety of factors, Some 


- out-of- 
excused from the test Ee 


ing program if their were 
school employment made it necessary, as 


ish 
also foreign Students whose knowledge of Engis 
was not sufficient to complete the tests. APP™ the 
mately 200 students were eliminated НОН. 2 
Study because they had not been enrolled ster 
high Schoo] during the preceding semes 
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TABLE т 
2 STUDENTS INCLUDED IN THE STUDY 
Ma i 
o wish "s | ale Students Female Students | Total 
counseled? Not N 
Counseled Coynseled Counseled быа Counseled E 
Yes 
No | 113 63 тоў 41 220 IO. 
2 4 

Did tes е 9 о; 4 36 : 

id not answer question | 3 21 5 16 $ 37 


Others ийге айй 

test ы eliminated because their criterion ог 
идеп a were incomplete. In all, 466 of the 951 
Ё s were Ыру" м = ал ae 
tion. not used in the present investiga 


The Test Instruments 


Pan following tests were used: SRA 

(окы Mental Abilities; Intermediate 

| dni Su Preference Record-Vocational, 

hensio (6); Test of Mechanical Compre- 

Pues т, Form BB (1); and SRA Youth 
2 Моту, Form A (7). 


са of interest, mental ability, and me- 
they i ie i pg were selected because 
high scho judged to be typical of those used in 
Teliabilit 01 counseling programs. Studies of their 
the ane an validity have been reported in 
For each ia? literature and publishers’ manuals. 
Well as Md ше tests a listing of references, as 
the three сш critical reviews, are included in 
urement ye T recent editions of Buros’ meas- 
Sufficient? js сок (2, 3, 4). All appear to be 
investip; y reliable for the purposes of this 
TI gation. 

tion за of the tests used in this investiga- 
are gener, ns a general comment. Although they 
in one S y accepted as being valid instruments, 
ünrelate 1 their validity may be considered as 
is sisted До the study. In this instance the study 
Counselin as dealing with the question of whether 
Counselec increases the congruence between the 

5 self-rating and his score. In other 
son his score so 


that Cs counseling teach a рег 
it he r in the form of a self- 


ur 

us outdoor interests. The fact that the 
t eg accepts the score as an indication of 
е cou interests and interprets it as such to 

lends credence to the idea that i‏ ا 
Ore E is effective counselees should show‏ 

Ores p between their self-ratings ай 

inpar sts of ош interests after counsel- 

ап they did before. 

With ea terion used in this study is СО 

e observed change when the agre 


Se 


ncerned 
ement 


of precounseling score and ratings is compared 
with the agreement of postcounseling scores and 
ratings. Essentially the criterion is anchored to 
agreement of score with rating, rather than 
agreement of the rating with some absolute or 
perfect measure of the characteristic upon which 
the counselee rates himself. If in counseling the 
score is interpreted as being indicative of a more 
perfect measure of the characteristic (whether 
or not it actually is is beside the point) then the 
criterion should be influenced in the hypothe- 
sized manner implied in the two major questions 
stated previously. 

The validity of the SRA Youth Inventory is 
not germane since this check list was not used as 
one of the variables in the criterion, It was em- 
ployed, rather, in an attempt to identify differ- 
ences between persons who ranked high and 
those who ranked low on the criterion, In a 
sense, an attempt Was made to validate it against 


the criterion. 

All the raw scores 
verted to quintiles on t 
norms with the excep 


on the tests were con- 
he basis of the published 
tion of the mechanical 


ability test. Here no appropriate norms were 
available, hence quintiles were based on a 
sample of students making up the population 


used in this study. 


The Ratings 
The self-ratings were secured by means 
of ascale designed for the study. Students 
o rate themselves on a five- 
step scale for each of the ten interest 
areas measured by the Kuder Preference 
Record, and also their mental and me- 
chanical ability. The interest or ability 
to be rated was described in nontechnical 
language. 
Students were asked to complete the 
rating scale at the beginning of the first 
In the last week of the 


testing session. 
summer term, seven weeks after students 


completed the first rating scale, a sec- 


were asked t 
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опа one was administered. It was identi- 
cal with the first except that the instruc- 
tions were changed to include a rationale 
for the request for repeat ratings, The 
second ratings were made during regular 
class periods. 


The criterion investigated in this study was 
based upon these self-ratings. Such ratings have 
been utilized in a variety of research studies and 
have themselves, in some instances, been sub- 
jected to study. The literature pertinent to the 
topic “ratings” is voluminous; that pertaining to 
self-ratings is more limited. It was recognized 
that self-ratings had a limitation in terms of 
reliability. Other measures might have greater 
reliability but none seemed to hold as much 
promise for studying changes in self concept as 
did self-ratings. Indeed, the task of rating onc- 
self requires one to reveal aspects of one's self 
concept or to resort to some sort of deception 
in order to conceal it. This is a compelling 
reason for using changes in self-estimates as а 
criterion of counseling. But an even more 
pertinent reason is to be found in the very un- 
dependability of the ratings. One of the objec- 
tives of counseling is to eliminate or at least 
reduce the influence of factors which contribute 
to the unreliability of ratings, Counseling should 
improve the accuracy of the counselee's self con- 
cept. If it does, it would make it possible for 
him to rate himself more accurately, 

At the outset of the study another limitation 
of the criterion was recognized: there is more 
involved in self-rating than just knowledge of 


FROEHLICH 


self. A counselee may, for example, "know" on 
the intellectual level that his performance on 
а test of mechanical comprehension is equiva- 
lent to the second centile of entering students 1n 
an enginecring school. But, if he drcams of 
bridging the Bosporus, irrigating the Sahara, or 
inventing a space ship, he may not really believe 
his mechanical comprehension is limited. His 
concept of himself as a successful engineer, 
figuratively speaking, insulates him from ше 
threat of accepting the meaning of his low mc 
chanical comprehension test score. The net result 
is that he rates his mechanical comprehension 
higher than his score rates him, The implica- 
tion of this limitation for this study is that thc 
criterion is not a measurement. of changes of 
actual self concept but rather is assumed to be 
a reflection of those changes. Р 

The reliability of the ratings used іп Hus 
study was estimated from data collected in the 
same school situation but in a different summer. 
Students were asked to rate themselves and then 
a week later to repeat the process. No tests were 
given, nor were interviews held during the inter- 
val between ratings. The reliability cocllicients 
reported in Table 2 were computed by using the 
customary formula for product-moment correla- 
tion and were corrected for coarse grouping "d 
the manner described by Wert, Ncidt, ane 
Ahmann (10). All of the uncorrected coefficients 
are significant at the оз level except that for 
“persuasive,” Of the 13 corrected coeficients, 9 
are significant at the .o1 level, These coefficients 
seem to indicate that the ratings used in Eris 
study have a degree of reliability not unlike that 
reported in other studies of ratings. Truc, they 


TABLE 2 
RELIABILITY COEFFICIENTS OF RATINGS - 
Correlation of Ratings d 
Aves N Uncorrected Corrected 
Reliability | Level | Reliability | Level of 
1 Level of Reliability Leve " 
Coefficient Significance Coefficient Significant? 
Interests: m 
Outdoor 31 -40 5 
Mechanical 31 “88 p po t 
Computational 31 .62 Jor 6 оз 
Science 31 E 5 р 5 
^ 39 +05 2 pes 
Persuasive 3o -25 18 g! б 
Artistic E ; То: б; н 
1 43 -os 6 Du 
Literary зо -74 ої s eT 
Musical . 31 -64 Тот -o1 
Social Service 31 51 “бї .68 I 
" x «5I ш 
Clerical 31 -38 05 ai +05 
ABS: | 
Menta 27 -78 
ee .Or 8 | 01 
Mechanical 30 . ук 
Me 46 E ә | о 
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are not as high as might be desired. Nevertheless, 
as the writer stated carlier, ratings appear to him 
з а potential for assessing self concept and 

5 Was an important factor in choosing them. 


The Counselors 


i The persons who did the counseling in 
t Ё 
he Demonstration Secondary School 
vere all graduate students specializing 1n 
St x ” 
аа personnel and counseling psy- 
© s Н 
ae in the Department of Education, 
Ü niversity of California. At the time of 
heir participation in this study, the 
с = E Р а 
ounselors were enrolled in the university 
su є x : 
mmer session working toward ad- 
у в "Е 
тея degrees in counseling psychology. 
Ё uring the preceding semester nine had 
e а 
еп employed as secondary school teach- 
ers, six as 
; 5, six as counselors at the college level, 
Our as i 5 : 
sd г as teachercounselors in high 
sch. ч ч M 
d ools, three as full-time counselors at 
CN DC MN . 
‘ ae level, two as school social 
vorkers а 
ЖУ] ers, one as teacher-psychologist, опе 
as ? $ x 
head counselor, and one as guidance 
Supervisor, 
The 
houen eee chose their own orientation to 
E None was completely nondirective. 
been Ron aware of the fact that the pupils had 
When i d first of the availability of counscling 
Promis nD reported to the testing sessions. The 
lest is opportunity to learn and talk over the 
iin is was used undoubtedly by the ad- 
Series ed аз a motivating factor during the 
Pupils of five testing sessions. Hence, some 
5 who reported to the counseling office 


came f, 
М ог й-б» 2n Sivan 
lion, € the express purpose of test interpreta 


Who ed ee made an effort to identify pupils 
relationship. inadequate notion of the kind о 
establish Pi that the counselor was willing to 
Certain thi hey made a special effort to make 

at all counselees realized that the 


couns 

Selor was will; Е + 

than a Was Willing to talk about things other 
test scores, 


T Р 
һе Criterion 


bein ouneeling practice, as has beet 
iS to hel out, one of the major sies 
Edge or id the client improve Ды ма 
that in. ^ The assumption 15 made 
Which ir a client to make wise choices 

nvolve him, it is necessary for 


him to know himself. In some counselors' 
orientation, the client can best discover 
sel-knowledge in a nondirective ог 
client-centered relationship. Other coun- 
selors consider it a more meaningful ex- 
perience if the counselor plays a more 
active role. Although counselors vary 
greatly in their relative adherence to 
either of these two points of view, there 
is rather general agreement among them 
that an important outcome of counseling 
is an increase in the client's knowledge of 
himself. The criterion investigated іп 
this study was designed to measure this 


outcome. 

The criterion was change in agree 
ment between a person's rating ol his 
ability or interest and an objective meas- 
urement thereof. In short, the client's rat- 
ings on à five-point scale in each area 
were compared with quintile ranks based 
on his test scores in corresponding areas. 
In terms of the desired outcome—im- 
provement in knowledge of sel[—certain 
assumptions had to be made concerning 


the criterion data. 


Although it was recognized that the test scores 
were fallible and perhaps might not be accurate, 
assumed that they did reflect an image 
of the client. From this assumption it followed 
that if the client had accurate knowledge of self 
his view of himself would correspond with the 
test-revealed one. Conversely, it was expected, 
therefore, that clients would learn about them- 
selves from test scores. 

A second assumption was made that if a 
person knew what his tested abilities and inter- 
ests were he would be able to rate them ac- 
Conversely, if he did not know what 
then he would be unable to 


it wa 


curately. 
his abilities were, 
e them accurately. 
The third asumption was made that al- 
though the subjects of this study would un- 
doubtedly have à considerable amount of in- 
formation about themselves, the process of test- 
1 counseling would improve this knowl- 


taking anc 1 Ў 
edge. These three assumptions were basic to the 


design of the study. 
presented for three groups 
ounseled, counseled-with- 


rat 


The data are 
of students: C 


out-test, and not counseled. 
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In the counseled group were included those 
students who were counseled and with whom the 
counselor discussed the results of the test. It 
should be noted that the counseled group varied 
from test to test; that is, not all persons who 
were counseled had all test results interpreted 
for them. One counselee, for example, may have 
had the interest inventory and mental ability test 
interpreted while another may have discussed 
the interest inventory and the mechanical apti- 
tude test with the counselor. Both counsclees 
would be included in the counseled group for 
the ten scores of the interest inventory. The first 
counselee would be included in the counseled 
group for the mental ability test and in coun- 
seled-without-test group for the mechanical apti- 
tude test, Conversely, the second would be in- 
cluded in the counseled group of the mechanical 
aptitude test апа counseled-without-test group 
on the mental ability test. 

The second group, counseled-without-test, in- 
cludes those persons who were counseled but 
for whom the particular test was not interpreted. 
Noninterpretation of tests was occasioned by 
several factors such as a delay in scoring the test 
50 that it was not available at the time of 
counseling or the judgment of the counselor that 
the test score would not make a significant con- 
tribution to the counseling. - 

The third group, "not counseled," includes 
those individuals who took the tests, but who 
were not counseled, Counseling was available 
for all who desired it but no effort was made to 
induce students to seek counseling. Of the 116 
students who indicated at the first testing ses- 
sion that they did not wish to be counseled, only 


17 later changed their minds and sought 
counseling. 


FINDINGS 


The data concerning each test were 
separated into three groups according to 
whether or not the individual was coun- 
seled, counseled-without-test, or not 
counseled, In order to study the agree- 
ment between scores and ratings, each 
individual’s score and corresponding rat. 
ing were compared. If both fell on the 
same numerical step of the five-point 
scale, it was tallied as agreement. That is, 
if the individual's score was a “five” and 
his rating a “five,” the values were con- 
sidered in agreement. But if his score was 
“five” and his rating “four,” the values 
were tallied as disagreement. The agree- 
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ment between first rating and score was 
determined, as was also the agreement 
between second rating and score, Each 
individual on the basis of agreement be- 
tween score and ratings of each area 
was classified into one of four categories, 
namely: 


Category 1. The score and first rating dis- 
agreed and so did the score and 
second rating, Persons їп this 
category consistently rated them- 
5 inaccurately, | 

Category 2. The score and first rating agrec¢ 
and so did the score and second 
rating. These individuals Зах 
consistent in rating themselves 
accurately, . " 

Category 3. The score and first rating dis: 
agreed, but the score and the 
second rating agreed. Included а 
this group are those individuals 
whose first rating was incorrect 
but who correctly rated them: 
selves on the second rating. ‘They 
changed from “disagree” t 
“agree.” | Е 

Category 4. The score апа first rating agree 
but the score and second ge 
did пог, These persons нөнү in 
their rating from an accurate p. 
to an inaccurate onc. They wen 
from “agree” to “disagree. 

Percentages based on the data tabu- 

lated according to the above-described 
categories are shown in Table 3. It is ob- 
vious that the majority of individuals 
ratings consistently disagreed with or 
score. This does not mean that they v 

+ . igh 
Not change their ratings. They mg) б 
have rated themselves too low on the firs 

х м г in 

rating and too high on the second; d 
both instances score and rating disagree: 
н А гу 

The median percentage for this сигар 
ror the 

for the counseled group was 55, for t i 
counseled-without-test group was 54, ane 

for the not-counseled group was 61. 

j " a 
5 These median percentages are indicative ОЁ is 
Situation of vita] concern to counselors. Why ê 
t that the majority of pupils in this study 
unable to rate accurately their abilities on € 


such a gross scale as the five-point one US 
here? In a Preceding section it was pointed 0 


B. 


ta —— 
۴ pe 
аза 


| 
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TABLE 3 


PERCENTAGES FOR CATEGORIES OF 


AGREEMENT BY GROUPS AND AREAS 


Counseled Counseled-Without-Test Not-Counseled 
Con- Con- = Gon a 

8 Con- | Dis- | Agree Sa | Con- | Dis- | Agree 8 Соп- | Dis- | Agree 
Area ay sist- | agree | to | y cay | sist- [agree | to | x dy sist- | agree га 

Dis. | ently | , to Dis- Dis | ently | to | Dis- Dic | ently | to | Dis- 

agree Agree | Agree | agree agree Agree | Agree | agree agree Agree | Agree | agree 

Interests: 
Outdoor 175 | 69 12 13 06 66 | 64 ır II 14 | 229 2 o8 zs 35 
Mechanical 174 | 55 17 13 15 66| 64 20 | o8 | o8 | 230] 70 Ir тї o8 
Computational 175 | so 15 17 18 6б| 43 26 17 14 | 22 6o 14 14 16 
Science едо | Se | zo | и | 66| sr | ts | 15-| 37 | aso] $8 | 25 | ar | as 
Persuasive 3) 8 | | 20 | e | 66] е | z& | 38 | ta | 290] 65 | за | 13 | за 
Artistic 175 58 20 1I Ir 66 | 46 17 20 17 230 | 47 19 16 18 
Literary WS) $3 | 22 | as | om | 66| ss | 23 | 34 | 99 | 220] so | 122 ы | чу 
Musical ms) $$ | ax | as | | $6| a7 | за | 35 | 24 | 330) gr | 29 | 12 | ài 
Social Service 175 55 I4 19 o9 66 | 45 18 17 20 | 220 63 II 12 14 
Clerical 175 | 56 17 12 15 66 | 45 14 23 15 | 230] бї 16 12 тї 
Abilities: 

Mental 150 | 68 15 п 06 75 | 66 17 оо o8 | 218 | 66 п 10 13 
Mechanical 79 6o 14 15 II 158 62 13 10 15 229 | бї I3 14 12 
Lusical 99 | 57 19 10 14 | 128 | 67 16 o9 o8 | 212 | 64 16 o8 12 


that self- itings were used because they required 
the subject to reveal aspects of his self concept 
Or resort to deception. The data in this study 
cannot yield the reason for the inaccuracies. 
"They do, however, hint at the possibility that 
lack of agreement between rating and test score 
Was brought about not so much by a lack of 
"knowledge" of self as it was by a lack of 
"acceptance" of the knowledge or an unwilling- 
ness to reveal the knowledge through ratings. 
Consider, for example, the rating of mental 
ability. АП of the subjects in the study had had 
extensive experience in school which provided 
many opportunities for them to acquire a sound 
basis for judging their mental ability. School 
marks, teacher statements, feelings of success 
and failure, parent judgments, and peer evalua- 
tons are illustrative of the experiences which 
Would help a person acquire an accurate self 
concept, Prior to counseling, 79 per cent of the 
Subjects inaccurately rated their mental ability. 
After counseling, when they would have been 
given the knowledge necessary to rate themselves 
ee 68 per cent persisted in rating them 
inaccurately. 


Comparison of Interest Profiles 


"E Kuder Preference Record— 

onal, Form С yields ten scores, 

indicative of interests in а particular 

E Counselors usually interpret ! ind 
кыш scores in the light of the total pro 
е. 


Voca- 
each 


i at 
Thus, one student's highest scores might be 


t i eas, 
E 85th centile in the clerical and et in 

© anot 's highest scores 
her student's high entiles of 


е ү 
Same area but are equivalent to © 


бо. In both cases, the counselor would probably 
interpret the highest scores as indicative of 
areas of greatest interest. But the interpretation 
in both would continue to the point of helping 
the client see the strength of his interests in 
relation to those of persons in the norm group. 
Essentially, profile interpretation is a process 
of discerning the strengths of interest in the 
ten areas measured by the Kuder in relation to 
each other and to the normative population. 
Because such interpretations were common prac- 
tice among the counselors included in this study 
it appeared desirable to consider the accuracy 
with which students reported their obtained 
Kuder scores when taken together as a profile. 
To make the profile comparisons, the data were 
reprocessed using à technique of assessing simi- 
larity between profiles (5). The technique is 
illustrated by the following data concerning 


one student: 


Kuder Interest Areas 


Column o raga Wu © £u 9 
First rating 5 g 5 2 4 4 4541 
Test score 3 4 5 2 5 2 ey Ж. 
Difference 2 —? oe =: 10 seio 0 6 
Square of 
difference 4 409 1 4 Qt g4 o 10 
“ou t) column, the “5” 
the "o (Outdoor Interest t 
x the student's rating of his outdoor interest 


i taking the test. The “3” 
ich he made before У Ў 
P ins quintile category based on his obtained 
ies score. The “o” is the difference when the 


i ubtracted from the rating. And 
di p "4" is the square of the difference 
is nd rating. The data for the other 


veen score а ue 
ipe: treated in a similar manner. A figure 
area 
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of 18 is obtained by adding the numbers in 
bottom row above. The square root of 18, or 
4-243, was used as this student's first-rating- 
profile score, 


Another profile score was obtained for 
this student by comparing the interest 
ratings he made after counseling with 
his test scores. The technique used fol- 
lowed the pattern described above. This 
process yielded a second-rating-profile 
score. In the case of the student cited 
above, his second-rating-profile score was 
3-464. Similarly, the data concerning all 
other students in the study were proc- 
essed to yield for each student a first- 
rating-profile score and a second-rating- 
profile score. 

Then the second-rating-profile score 
was subtracted from the first-rating-profile 
score. In the case of the student used as 
an example in the preceding paragraphs 
this computation yielded a positive num- 
ber, 779 (ie, 4.243 — 3464 = .779). 
The algebraic difference between the pro- 
file scores indicated whether the indi- 
vidual had moved toward greater con- 
gruency between the profile of ratings 
and the profile of scores, or toward less 
congruence, A positive difference be. 
tween the profile scores indicated that 
the second profile of ratings согге- 
sponded more nearly to the obtained pro- 
file of scores than did the first profile of 
ratings. Conversely, a negative difference 
means that the first ratings were more 
consonant with scores than were second 
ratings. In fine, positive differences 
showed increased "accuracy" and nega- 
tive differences a deterioration in “ac- 
curacy.” 

Separate frequency distributions of the 
differences between profile scores were 
made for the counseled, counseled-with- 
out-test, and not-counseled groups. The 
distributions ranged from high positive 
through zero difference to high negative, 
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The zero-difference category was not in- 
terpreted as indicating that the student 
did not change his profile of ratings. The 
zero difference merely indicated that the 
subject’s second profile of ratings agreed 
neither less nor more closely with the pro- 
file of scores than did his first rating pro- 
file. 

For the distributions of the differences 
between the first-rating-profile score and 
the second-rating-profile score the means 
and standard deviations were computed. 
Table 4 contains these data. The means 


TABLE 4 
MEAN AND STANDARD DEVIATION or 
DIFFI ILE SCORES 
Com | мө 
К Coun- | Ar Coun- 
Statistic sed 1 With- 1 
selcd out sele! 
Test = 
Mean of difference .41 .26 | -04 
Standard error of 07 
mean . 09 EU i 
Standard deviation 1.09 -95 50 
Number of cases 175 66 23 ын 
-— 


varied in magnitude in the expected 
manner; namely, the counseled group 
showed the most change in the direction 
of greater agreement, the not-counselet 
group the least change. A t test жш 
that the means of the counseled and dis 
counseled-without-test groups аге n 
cantly different from zero. In conside! 
ing the mean difference of 41 2. 
should not overlook the fact that the we 
ference between score and rating о, 
not exceed 4.00. Hence the difference ^ 
slightly more than 10 per cent of ine 
largest possible difference. To test tP 
Significance of the difference petuo 
means, { ratios were computed. The i 
Tatio between the mean of the counsele 
Broup and the mean of the noncounsele* 
8roup was 3.52, significant at the 2 
level. The t ratios between the mean 


—— 
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TABLE 5 


COMPARISON OF CERTAIN CHARACTE 


RISTICS OF SUBGROUPS SELECTED ON BASIS OF ACCURACY 


or RATING OF INTEREST 
Counseled Without T 
Counseled Interest Test Not Counseled 
Variable = aa = 
Deteri- Deteri- Deteri- 
азе ai а 
Increased ота Increased А Increased arated 
—T А | 
Mean age 15.8 15.0 16.1 16.3 16.1 ї ә 
Y ean school grade 10.4 10.6 10.7 10.6 10.3 10.3 
Mean Р.А score 3.3 3:5 | 3-1 2.$ л 31 
Mean number of tests inter- 
_ preted sa | 4-7 3.7 3-5 е E 
Number of: males 23 24 п 7 30 38 
females 24 23 7 11 26 2 
Number: 
desired counseling | 45 45 15 тал 25% 3o 
did not desire counseling | 2 2 3 r | 28" 23" 


^ Some students did not respond to t 
counselor?" Therefore 
nonrespondents was ехс 


the counseled-without-test group and the 
counseled and the not-counseled groups 
were found to be 1.03 and 1.63, re- 
spectively. Neither was statistically sig- 
nificant. 

The data thus far reported have an- 
swered affirmatively the two questions 
Spelled out in the section which dealt 
the purpose of the study. They re 
f к ed that there was a significant dif- 
кы between precounseling and post- 

nseling criterion measurements. The 
data also revealed that these observed 
differences were in the direction of more 
oe after counseling. The st 
sty у of these differences was also demon- 

rated by the differences between the 
iis counseled-without-test, and 
otcounseled groups. The neat stair-step" 
Jes of mean differences in Table 4 from 
allest for the non-counseled group to 
‘rgest for the counseled group lends cre- 
dence to the belief that the observed 
“anges are in part at least а function of 
Counseling. 


Co = 
Mparison of Factors 


A ? s 1 
In this section data will be presente“ 


he question, '^ 
these totals differ from others in the table. In 
ive, tests of significance were not applied. 


an opportunity to talk with а 


“Would you like 
instances where the number of 


concerning factors thought to be associ- 
ated with changes measured by the cri- 
terion. They are: age, grade placement, 
intelligence, sex, number of tests inter- 

preted, and desire for counseling. 
In order to determine if the factors 
were associated with the criterion, the 
were divided on the basis of the 


subjects 
difference between the first-rating-profile 


and the second-rating-profile score. 
ent of the students with the 
largest positive differences (increase in 
“accuracy’’) and the.27 per cent of the 
students with the largest negative dif- 
ferences (deterioration in "accuracy") in 
the area of interests were identified in the 
counseled, counseled-withouttest, and 
not-counseled groups. This process 


yielded the six subgroups. — — 
ach subgroup the statistics sum- 


score 
The 27 per c 


For е 
marized in Table 5 were obtained. When 
а 
appropriate tests of significance were 


differences between the “in- 
creased” апа the “deteriorated” sub- 
the counseled, counseled- with- 
out-tests, and not-counseled groups, none 
of the differences was found to be sta- 
tistically significant. Comparable results 


applied to 


groups in 
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TABLE 6 
NUMBER OF PROBLEMS CHECKED ON THE SRA Youth Inventory 


Counseled Counseled Without Test Not Counseled 
i i d Deteriorated 

Increased | Deteriorated Increased | Deteriorated Increased | Deteriora r 

Inventory Areas N=34 N=29 sto N-n N12 |r ratio N-ss N-ss tratio 

Mean| SD |Mean| SD Mean| SD [Mean | SD Mean| SD |Mean| SD > 

My School 6.00 | 4.34 | 5.31 | 3.48 68 | 6.64 | 3.87 | 4.92 | 3.17 | 1.20 | 7.25 | 4.40 | 6.44 | 4-50 | «94 

After High И 2 .68 

School 11.68 | 7.67 | 8.97 | 6.09 | 1.54 [13.14 | 7.56 9-25 | 7.43 | 1.27 | 0.0 7.65 woe 
About Myself 6.70 | 6.54 | 6.14 | 4.89 -38 | 8.86 | 5.29 | 6.75 | 8.70 | +70) 7.89 | 7.84 

Getting Along а 1B 
with Others 6.79 | 6.42 | 6.93 | 4.50 то | 8.29 | 6.17 | 4.67 | 5.02 | 1.50 | 6.06 | 6.36 

My Home and 6 +07 

Family 4-18 | 6.12 | 3.52 | 5.35 45 | 4-36 | 3-20 | 2.50 | 3.23 | 1.05 5.11 | 6.4 ue 

Boy Meets Girl | 3.or 3-93 | 3-93 | 4.30 02 | 3.57 | 3.81 | 2.92 | 1.89 54 | 3.56 | 4.77 "74 
Health 2.62 | 2.29 | 2.79 | 2.59 2 2.57 | 2.82 | 2.33 | 1.97 2 3.05 | 3.14 

"Things in á zin 
General 4-18 | 4.34 | 3.66 | 3.17 54 | 4-21 | 4.99 | 4.00 | 3.51 38 | 3.93 | 4.37 

Basic Difficulty 1.08 

Score 12.41 |I1.24 |11.66 | 8.58 +30 [14-14 | 7-94 |12.50 |13.50 +35 [15.05 |12.76 | 


were found 
mechanical 
treated. 

On the basis of these data, it was con- 
cluded that the improvement or deteri- 
oration in the "accuracy" of students' self- 
ratings apparently was not a function of 
their age, school grade, mental ability, 
sex, desire for counseling, or number 
of tests interpreted during counseling. 
It appeared, therefore, that the criterion 
employed in this study will be useful in 
the further study of counseling because 
it was not influenced by Bross factors 
over which counselors have little con- 
trol. This conclusion led to a desire to 
learn even more about factors associated 
with the criterion. One attempt to study 
such factors will be reported in the next 
section. 


when mental ability and 
aptitude were similarly 


Comparison of Problems 


The SRA Youth Inventory appeared 
to be an instrument which might be used 
to identify some of the more subtle 
characteristics which distinguished those 
students who show more agreements be- 
tween scores and ratings after counseling 
than before. The Inventory is a list of 
298 problems of which the testee checks 
those which apply to him. The more 
problems he checks the higher his score. 


Did students who showed more арата 
have more or fewer problems than those ES 
showed fewer agreements after counseling? ы a 
answer to this question was sought in a COH 
son of the SRA Youth Inventory scores of We 
students who showed the greatest increase em 
agreements with gg students who showed Eos 
greatest deterioration in the agreement Бера 
their profiles of interest scores and ratings. Т ne 
students were drawn from the upper and Јаке 
27 рег cents of the distribution of differences 
between first- and second-rating-profile pode 
previously described. The mean inventory SCO 
of the "increased" group was 46.92 with 1” 
standard deviation of 30.32. The “жини, 
group had a mean inventory score of 41:725 p 
standard deviation was 29.64. The t ratio Я 
the difference between these means was 1.22, ae 
significant. Hence, the question asked above К 
answered: there was no difference between s 
Broups in terms of the total number of problem 
checked on the Inventory. or 

The next Step was to determine неше. 
not there was a difference between the їп: 
creased” and “deteriorated” groups in the pA 
ber of problems checked in specific areas. Т 
SRA Youth Inventory, as previously pointed wb 
yielded scores in eight areas of adjustment. ps 
order to further refine the analysis, the groupe 
Were broken down into counseled, counsel 
without-test, апа not-counseled subgroups. I 
mean number of problems checked by cach su n 
group for each area is presented in Table 6. es 
Table 6 the ¢ ratios based on the differen? 
between means are also given. None was Юрт: 
to be statistically significant. Hence, it was СО! s 
cluded that the SRA Youth Inventory € 
did not identify those students who mon 
more agreements between their profile of sco” 
and ratings after counseling than before. ү 

The two comparisons based on number he 
Problems checked did not take into account t 
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possibility that certain particular problems 
might be checked by the "increased" group and 
others by the "deteriorated" group. The previ- 
ously discussed analyses were based simply upon 
number of problems checked. Hence an item 
analysis was undertaken. The groups were di- 
vided into random halves. The resulting groups 
were (a) Increased I, N = 50; (b) Increased 11, 
»; (c) Deteriorated I, N = 49; and (d) Dc- 
ted II, N = 50. The item responses of the 
Increased I and Deteriorated I groups were 
tabulated and percentage of response for cach 
group for cach item was computed. If the differ- 
ence between the percentages for the two groups 
was significant at the .05 level, the item was 
retained. 

There were 35 of the 298 items in the test re- 
tained. Of these, 32 were keyed for increase and 
3 keyed for deterioration, These items were cast 
into a scoring stencil and all answer sheets for 
the test were rescored, A single score was ob- 
tained by subtracting the deterioration-keyed 
items from the increase-keyed items. The mean 
scores and standard deviations were computed. 
These data are presented in Table 7. The differ- 
ences between the means of the Increased 1 and 
Deteriorated I groups yield a ! ratio of 7.58, 
highly significant. This was expected since the 
scoring key was based on these groups. In an 
attempt to cross validate, the 35-item scoring 
key was applied to the answer sheets of the 
subjects included in the Increased II and De- 
leriorated II groups. The data in Table 7 con- 


TABLE 7 


MEAN AND SrANDARD DEVIATIONS ON SCORES 
DERIVED FROM ITEM-ANALYSIS KEY 
(Cross- Validation Sample) 


Stand- 


Group N Mean S. 

tions 

Increased I 50 8.00 4.56 
Deteriorated I 50 2.51 2.20 
ncreased II 50 4.76 2.84 
Deteriorated IT 4o | 5-18 | 3-54 


cerning these two groups indicated that the 35- 
item key did not hold up. When a ¢ test was 
applied to the difference between the means of 
these two groups, it yielded a t ratio of .65, 


NOt significant. 


These analyses of the SRA Youth In- 


ventory data suggested that the increased- 


deteriorated classification was not à func- 


tion of the number or kind of problems 
the student checked. 


SUMMARY AND DISCUSSION 

The over-all purpose of this investiga- 
tion was to study a proposed criterion 
for the evaluation of counseling, namely, 
change in agreement between self-rating 
and score, It is appropriate to ask, “What 
was learned about the criterion?" De- 
tails in the answer to this question can 
be found in the preceding pages. They 
may be summarized briefly. The criterion 
data varied in the expected direction, 
that is, the findings followed the logic 
of improved self-knowledge after counsel- 
ing. From this point of view the cri- 
terion which was adopted appears to 
hold promise as a useful indicator of 
counseling effectiveness. 

Its potential usefulness was further 
substantiated by the findings that the 
criterion appeared not to be affected by 
age, sex, school grade placement, intel- 
ligence, or desire for counseling. These 
are factors over which a counselor has 
no control (except by limiting his prac- 
tice to specified types of clients). The 
important implication of these findings 
is that the criterion variable is independ- 
ent of such factors which are extraneous 
to the counseling process per se. 

A related finding was that neither the 
number nor the kind of "problem" 
which counselees verbalized on the prob- 
lem checklist was associated with the 
criterion variable. If this result were ac- 
cepted at face value, it would do vio- 
lence to beliefs which counselors have 
about their relative effectiveness with 
clients having "minor" as opposed to 
“major” problems of adjustment, or 
with clients having certain kinds of 
problems (e.g. vocational choice as op- 
posed to feelings of inferiority). In con- 
sidering these data, it is pared да 


— 
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keep in mind that the criterion involved 
ratings in the areas of ETE and 
ability. It was not concerned with so- 
called problems of personal adjustment, 
the forte of the problem checklist used 
in this investigation. But even in areas 
of the problem checklist closely related 
to interests and ability, no statistically 
significant association with the criterion 
was noted. 

This brief synopsis of the findings of 
this investigation can serve as a starting 
point for considering next steps. 


The criterion explored in this investigation is 
an “immediate” one, i.e., it is available within 
a short time after the experiment is completed. 
But its relationship to the long-range objectives 
of counseling has not been established. Herein 
lies a great weakness. If one were to argue that 
self-knowledge is the object of counseling, the 
criterion studied might be presumed to be a 
reasonable measure. However, few counselors, if 


any, would be satisfied with self-knowled 


ge as an 
end in itself, 


The common expectation is that 
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self-knowledge should influence counselee bg: 
havior. Here the criterion is silent, for it docs 
not measure behavior, And in this silence 15 
found an important gap in the usefulness of ш 
criterion, as well as a suggestion for additiona 
research. In future research it might be possible, 
for example. to discover that the criterion is 
associated with more intelligent, more construc- 
tive, or more adaptive postcounseling ен 
If such a discovery were made, then the criter ior 
would acquire sociological significance. an 

Another fact must be considered. Some stu 
dents who were not counseled showed as much 
change on the criterion in the direction of ШӨ 
"accurate" ratings as did students who we! 
counseled. Conversely, some counselces de: 
teriorated in the "accuracy" of their ratings 45 
much as some not-counseled students. Ун 
factors are associated with change as measure’ 
by the criterion? The design of the present Wo 
periment was such that only counseling as а 
process was studied. Are there variations SM 
the process from counselor to counselor. o 
selee to counselee, or problem to problem un 
account for the observed differences 1! MT 
criterion variable? If so, are these factors also Es 
work in different form among not-counscler 
students? These are crucial questions for whi 
no ready answers are available. 
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Ageression in Fantasy and Overt Behavior' 


ARTHUR R. J 


Teachers College, Columbia University 


uis PAPER is concerned with the re- 

lationship of Thematic Appercep- 
tion ‘Test responses to overt behavior. А 
review of the research on this subject (9, 
ch. 1) suggests the following generaliza- 
lions: 

а. The thematic content per se of 
TAT stories seems to be related. very 
slightly, if at all, to general behavioral 
traits or dispositions. But thematic con- 
tent does seem to be related to temporary. 
situationally induced. behavior, affective 
states, and drives. 

On the basis of correlations between the 
fantasy themes of 4o adolescent. boys and girls, 
and teachers! ratings of behavioral characte 
lics, Symonds concluded that t 
between these two sets of va 
significant and negligible" (19. P- ). Studies 
by Pittluck and Gluck (7) found no rclation- 
ship between fantasy aggression and general 
overt aggressiveness. On the other hand, Bellak 
(3) showed that the number of aggressive words 
in TAT stories increased when the storyteller 
Was insultingly cri cized for the low quality of 
his stories, Essentially the same finding was made 
by Feshbach (6), who also showed that Ss who 
had a fantasy outlet for their induced aggres- 
Sion (by writing TAT stories) expressed less ag- 


sed on à dissertation. sub- 
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the Joint Committee 0n Graduate Instruction, 
Columbia University. The author is indebted 
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atry (Maudsley 


gression in a sentence completion test designed 
to get at the Ss’ hostility toward the insulting 
examiner, Clark. (1) showed that male Ss who 
aroused by Dbe- 


presumably had been sexually 
ing shown photographs of nude women had 
fewer themes of sex in their TAT stories than 
did control Ss, while another group of Ss under 
the disinhibiting effects of alcohol produced 
more themes of sex in their TAT stories after 
having been shown pictures of nude women. 
Lindzey (11) demonstrated that extrapunitive 
behavior on the part of the hero in TAT proto- 
cols increased significantly following failure in 
a social situation, McClelland et al. (12) have 
reported substantial positive corr ations be- 
tween н Achievement in TAT fantasies. and 
several measures of behavioral achievement when 
the Ss were tested in an hievement-oriented 
situation, Sanford (16, 17), and Atkinson and 
McClelland (1) have shown that Ss produce more 
T when they are 


food-related responses to the © 
made hungry by food deprivation. 


b. The expressive or behavior-sample 
aspects of the TAT responses seem to 
be correlated with overt behavioral traits 
and would therefore seem to be a more 
valid basis for predicting overt be- 
havioral traits than are the thematic or 
fantasy aspects of the TAT. 


An example of this is Balken and Masser- 
man's study (2). showing language and stylistic 
differences between obsessives, hyster and 
anxiety states. On the basis of a study correlating 
ious aspects of TAT responses with various 
personality variables, Hartman stated: "Atten- 
tion in the past has been centered upon thematic 
analysis in ТАТ interpretation, Formal charac- 
teristics of TAT responses should be given in- 
creased emphasis inasmuch as they can be more 
objectively determined and may, particularly in 
application to group testing, be more revealing 
of certain aspects of personality” (8, р. 85). 


Problem 
The presen 
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thematic and the behavior-sample as- 
pects of the TAT and their relation to 
overt aggressive behavior as a general 
disposition or trait. Aggression was the 
variable chosen for study because it lends 
itself particularly well to this kind of 
investigation; it is more easily identified 
and can be more reliably rated in both 
the TAT stories and in the S's behavior 
than can many of the other personality 
variables purportedly revealed by the 
TAT. A review of the existing research 
led to the formulation of the following 
questions, in terms of which the present 
study was planned: 


1. Is there a direct relationship between ag- 
gression in TAT fantasy and in overt behavior? 
In other words, do Ss who show much overt 
aggression have more themes of aggression in 
their TAT stories than do Ss who show little 
aggressiveness? 

2. 15 there an inverse relationship between 
ntasy aggression and overt aggression? Both 
Symonds (19) and Sanford (18) have suggested 
that needs which are repressed in overt behavior 
are more likely to find expression in fantasy, 
and that needs which are worked out in behavior 
are less likely to be expressed in fantasy, 

3. Does the degree to which punishment is 
associated with aggression in fantasy permit pre- 
diction of overt aggressiveness? Some clinicians 
believe that when in fantasy punishment is as- 
sociated with aggression, the S is less apt to be 
overtly aggressive. A study by Mussen and 
Naylor (14) lends support to this hypothesis, 

4. The same idea has been put forth with re- 
spect to the amount of defense against aggression 
anxiety in the TAT. Is an $ who denies the 
aggressive implications of a TAT Picture or 
who has to rationalize or tone down the aggres- 
sive implications of his own Story less apt to be 
overtly aggressive than an $ who does not de- 
fensively modify his aggressive fantasies? Pitt- 
luck’s research (15) suggests an affirmative answer 
to this question. 

5. Do TAT stories when viewed as a behavior 
sample show a relationship to the S's overt be- 
havior in a wide variety of other situations? For 
example, would aggressively delinquent boys re- 
veal in their TAT productions the socially de- 
fiant, rebellious attitude which characterizes 
their behavior in other situations? Delinquent or 
antisocially aggressive behavior may be inter- 
preted partly as a rejection and defiance of cer- 
tain social standards (5). The TAT would thus 


far 


be expected to elicit from delinquent boys be- 
havior which violates the taboos of thc examiner 
or of the setting in which the TAT stories are 
produced. In other words, the TAT production 
may be viewed as a direct sample of the 5 s be- 
havior and not simply as a fantasy which in- 
directly reflects the S's needs and conflicts. 
According to this hypothesis boys who exhibit 
socially unacceptable forms of aggressive be- 
havior in school would give a sample of their 
"unacceptable" behavior in the ТАТ by using 
tabooed language, themes of sex, tabooed forms 
of sex, and unusually repulsive or taboocd forms 
of violence, such as mutilation of the victim. 
These forms of expression may be regarded E 
aggressive acting-out in the TAT situation and 
are therefore direct behavior samples. 


On the basis of these questions there 
were formulated the specific hypotheses 
which could be tested by comparing the 
ТАТ productions of three groups of Ss 
differing widely with respect to overt 
aggressive behavior. Each group was rela- 
tively homogeneous as regards aggre’ 
sive behavior, and the Ss in each group 
were selected as relatively extreme repre 
sentatives of a particular type of ре 
havior. It was believed that these condi- 
tions would allow a maximal opportunity 
for the relevant TAT variables to mant 
fest significant differences, It was com 
sidered important that all the Ss be of 
approximately the same age, be selected 
from the same setting, and be teste“ 
under the same conditions. These T€ 
quirements were fulfilled. The three 
BrOUpS, composed of high school boy’: 
were labeled 4 Egressive-Bad (socially ү 
acceptable overt aggression); Aggressive 
Good (Socially acceptable overt арте» 
Siveness); and Passive (extremely lacking 
In all overt aggressive qualities). T 
method of selecting the Ss is described 
in the following section, 


Hypotheses 


st 
1. The Aggressive-Bad group has the idt 
ire SI Sion; the Passive group has the Je 
(Direct relationship) 


2 st 
2. The Aggressive-Bad group has the lea 


AGGR 


TAT aggression; the Passive group has the most. 
зс relationship.) 

3. The proportion of mild forms of TAT 

ression (Scores 2-6) is smallest for the Ag- 

gressive-Bad group. (Qualitative relationship.) 

4. The proportion of stories in which the 
hero is the victim of aggression to those in which 
the hero is the aggressor is smallest for the 
Aggressive-Bad group, gest for the Pas ive 
group. 

5. The Aggressive-Bad 5s have fewer themes 
of suicide than either of the other groups. 

This is based on the belief that suicide repre- 
sents aggression turned against the self and that 
Nonaggressive Ss tend to be more intropunitive 
and develop reaction formations against the overt 
expression of aggressive impulses. 

6. The Aggressive-Bad group has the smallest 
Proportion of punishment relative to aggression 
in their TAT stories; the Passive group has the 
largest. proportion. (Dynamic relationship.) 

7. The Aggressive-Bad group has the smallest 
Proportion of defense relative to aggression; 
the Passive group has the largest proportion. 
(Dynamic relationship.) 

“Defense” refers to an evasion used by the 
Storyteller, presumably to avoid aggression 
anxiety. The aggressive fantasy is not expressed 
as ' ” aggression, but is rationalized, made 
socially acceptable, or accidental, or is qualified 
in such a way that its aggressive implications are 
lessened or hidden. 

8. The Aggressive-Bad group has the largest 
Proportion of aggressive stories containing “raw” 
aggression, i.e., aggression free from association 
with punishment (or guilt) or defense. This is 
a corollary of Hypotheses 6 and 7. (Dynamic 
relationship.) 

9. The Aggr ive-Good and Passive Ss have 
More themes of natural death than the Aggres- 
sive-Bad Ss. (Dynamic relationship.) 

This is derived from the belief that natural 
death occurring in a TAT story represents à 
disguised expression of aggression and is thus 
Psychologically similar to the above-mentioned 
defenses, 

10. The Aggressive-Bad group has 
themes of sex, tabooed sexual content, tabooed 
language, and tabooed or "shocking" forms of 
Violence than either of the other groups. (Be- 
havior-sample relationship.) 

This is based on the assumption that the 
Expression of sexual and tabooed themes in the 
ГАТ Stories, especially when they are written ME 
а School setting, represents a lack of inhibition 
about expressing socially disapproved thoughts 
and actions, Tabooed language and tabooed 
Sex are also forms of verbal aggression—pethaps 
Ше only forms of verbal aggressive behavior that 
the test situation permitted. (A more open form 
BE aggressive behavior would have resulted in 


more 
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the S's dismissal from the testing room.) When 
these tabooed ideas can “break through” into 
written. expression, it would seem to indicate 
both a certain preoccupation with the socially 
tabooed and a flaunting defiance of customary 
decency and propriety. The manifestation of 
these attitudes in the general school behavior 
of the Aggressive-Bad Ss was probably among 
the reasons many of them were selected by their 
teachers for this experiment. 


PROCEDURE 

Subjects 

Three types of Ss were desired, each 
showing different behavioral character- 
istics with respect to overt aggressive- 
ness. As a convenience, the three types 
were labeled Aggressive-Bad, Aggressive- 
Good, and Passive. 


The Aggressive-Bad Ss were those whose ag- 
gressiveness got them into trouble at school. 
They were overtly aggressive in disruptive, 
socially unacceptable ways. 

The Aggressive-Good Ss were those whose 
aggressiveness found socially acceptable expres- 
sion in athletics, leadership, competition, and 
other nondisruptive activities. 

The Passive Ss were those so lacking in ag- 
gressive qualities as to be shy, quiet, unobtrusive, 
unassertive, and diffident. 

Ss of these three types were selected in an 
all-boy high school with over a thousand stu- 
dents, most of whom live in an industrial com- 
munity. Every teacher in the school was provided 
with instructions for selecting these Ss. Each was 
asked to submit the names of five boys he 
knew from personal experience during the 
current school year (this was done in May) who 
most fully met the description of the specified 
types. Every teacher was asked to rank these Ss 
for degree of conformity to the behavioral traits 
described and also to fill out check lists of vari- 
ous descriptive criteria with respect to kinds of 
aggressive behavior, indicating those character- 
istics which best described his reasons for select- 
ing each S. 

Fifty-one teachers submitted a total of 756 
names. This represents only 433 students, since 
some were submitted by as many as six teachers. 
There were disagreements in classifying 25 of 
the students. These were immediately eliminated 
as possible Ss for the present study. T 

From the remaining 408 students a smaller 
number was selected for testing. The Ss were 
selected first on the basis of the number of 
teachers submitting their names. The second 
basis for selection was the rank assigned to the 
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S by the teacher. The behavior check lists were 
used to ensure a high degree of homogencity 
within each of the groups with respect to types of 
aggressive behavior. Items frequently checked by 
the teachers for Ss in each group were: 

Aggressive-Bad. Disruptive, unruly, bad 
temper, defiant of authority, headstrong, boister- 
ous, "tough," surly, destructive of property, ag 
gressive bravado. 

Aggressive-Good. Active, energetic, helpful, 
leader, likes competition, "carries the ball," self- 
assertive, acceptable verbal aggression (e.g, de- 
bate), seeks recognition, engages in competitive 
sports, 

Passive. Quiet, retiring, meek, unassertive, 
withdrawing, diffident, timid, does not actively 


participate, submissive, a follower, “shrinking 
violet.” 


It is believed that relatively homo- 
geneous groups of the most extreme and 
representative Ss of each type were ob- 
tained by this procedure. АП the Ss were 
between 15 and 15 years of age. Fortui- 
tous circumstances at the time of testing 
prevented the Ns in each group from 
being equal (Aggressive-Bad — 25, Ag- 
gressive-Good — 22, Passive = ә ) 


1 


Selection of TAT Pictures 


A pilot study was conducted in a com- 
parable all-boy high school in another 
city in order to determine the most ef- 
fective and efficient method of adminis- 


tering the TAT and the most suitable 
pictures for the purpose of this study. 
The Ss in the pilot study were students 
in four different English classes (total N 
= 00). Four different methods of group 
administration of the TAT were tried, 
Analysis of these data led to the follow. 
ing conclusions: (a) the Ss were most 
productive when required to write stories 
to five of the TAT pictures during the 
class period of 50 minutes; (b) there were 
no systematic differences in the stories (as 
scored for this study) between 10th-, 
11th» and igth-grade Ss; (c) all the 
stories were scored on the main variables 
of this study and from these data the 
TAT pictures were ranked according to 
their power to elicit these themes. The 


theme of aggression was weighted most 
heavily in determining the rank ien 
of the pictures, The ten most fruitlu 
cards, in the order of their effectiveness, 


are listed below. These were used in 
the main study. 
SBM ... Man—hands clutching him from be- 
hind. А 
13МЕ ... Woman lying оп bed—man_ standing 
near. : 
17 GE . Woman standing on bridge ove! 
water, cared 
18GF A woman has her hands squeeze 
around the throat of another Samum. 
ЗСТ . Woman standing in doorway with 
downcast head. - of 
1 ... Young woman clutching shoulders о 
young man who is turning PT 
3BM ... Boy huddled on floor against couch. 
СЕ . From behind a tree a young woman 
` watches another woman running 


along a beach. А 

- Man standing under street light. . 
‚ Gaunt man standing among grave 
stones. 


Administration of the TAT 


The Ss were tested in groups of about 
25 Ss of all types (Aggressive-Dad, AF 
gressive-Good, and Passive). Thus there 
were three test sessions. Ss sat in altel 
nate seats and were well proctored. Each 
session lasted go minutes and all Ss ^ 
ten TAT stories within that time. 
shortened and simplified version of X 
standard Murray instructions (13). W% 
printed on the cover of the test booklet 
in which the Ss wrote their stories; tes 
instructions were also read aloud by ШЕ 
examiner at the beginning of the sessio". 

Three sets of TAT cards were jenes 
the examiner exchanged each S's pict! 


as each Story was completed, or us 
themselves exchanged oe 


pictures. 
each $ could write at his own SP 


rouse 
though to ensure that everyone WO э 
а 

a 


вей, 


finish in one session, the examiner 
nounced at 8-minute intervals that 
certain number of stories should да 
been completed. An examination of Fi) 
mean scores on the TAT variables fo 


a 


9. 
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ены е three test sessions (9, p. 96) 
of энн Me if there was any “leakage” 
in the ae en about the TAT from Ss 
and {несер ED the Ss in the second 
results м“ rne it had no effect on the 
no vals Vd experiment. There were 
betw BISEGIE WE significant differences 
veen the three sessions. 


Scoring the TAT 
Taf cen scoring system was devised 
ам DOSS ol this study. The key 
ihe amd "Aggression, Punishment for 
Кайрап qui^ a A distinction was made 
acter) of q near the hero (central char- 
Victim of а stony B the aggressor or the 
fined mea i gonna Since a more re 
БНЫН of classifying aggression and 
Were келе desired, these variables 
according ivided and assigned “scores 
АП "e to their intensity. 
related. other variables are indirectly 
simply on ение T hey were scored 
the story the basis of their presence M 
у. Suicide is aggression against 


the : 8 
se T " А 
If. The occurrence of. № atural 


Death i 

hh. story шау be a disguised ex- 
anguage h.e Sex and Tabooed 
Ons ic E have aggressive connota- 
Sion in See Re an expression of aggres 
dicate qy SEAR: Also they may 
Approved з inhibition of socially dis- 
'iolence 8 of expression. Tabooed 
Sadistic : a particularly primitive and 
Scoring jr. The 

Tin 

Made were 
fquivo 


ti 


expression of aggression. 
& instructions, given below, 
as complete, explicit, and wr 
em cal as possible, so that by following 
in eta even persons not trained 
in ТАТ can achieve close agreement 
Scoring, 8 


ец 
Clions for Scoring the TAT 


Sco 

re 

OY, only what is explicitly stated in the 

Might MI make any inferences about what 
© implied. 


AND OVERT BEHAVIOR 


сл 


Ageression Score 


Agg 
Hero Aggressor. The main character of the 
story is responsible for the aggression; he (or she) 
is the one who can be called the aggressor. For 
determining the main character, see Murray's 
criteria for distinguishing the hero (13, p. б). 

Hero Victim, When the main character of the 
story is aggressed against by another character in 
the story, the aggression js scored in the Hero 
Victim category, since the hero is not the one 
who commits the aggressive act but is the vic- 
tim of the aggressive act. 

Indeterminate, When there is aggression in 
the story and it is difficult or impossible to 
decide if the story has a main character, the 
should be scored in the Indeterminate 


aggression 

category. 

о Inert, no action, mere contemplation; 
neutral or nonaggressive action; no indica- 


tion of aggression in any form. 

А character in the story dominates or 
controls another, takes self-initiated action, 
seems to do things with a purpose of his 
own. 
4 Character in the story is actively working, 

struggling, striving, willfully overcoming 

obstacles, resisting pleas of others, daring 
exploits, being actively brave, etc. 
6 Verbal aggression; swearing at another, 

“telling off” another, threatening another; 


yerbal coercion. 
g Physical agg" 


„ 


ession not resulting in death 
and which usually is not expected to cause 
death; fighting, struggling with another, 
hitting, causing injury, physically subduing, 
rape, robbery, ‘and deliberate destruction of 


property. . 5 
9 Physical aggression not resulting in death 
^ but which usually results in. death, such as 
oting another person, choking someone, 


ting any kind of injury which 
enough to result in death. 
liffers [rom 1o in that. killing 
t to kill or cause death, 


sho 
knifing, inflic 
seems severe 
(This score GU" 
or death, or the inten 


are not explicit.) | | 
Physical aggression resulting In death: 


murder, killing. (Killing ог intent to kill 


must be explicit.) 


10 


t Score 

There can be a Punishment score only if the 
Aggression score is 6 or greater. The punish- 
ment is always for aggression in the story. If a 
d cems to be punishment 


character receives What st | 
and there js no indication of previous aggres- 


on his part. then what seems like punish- 
red under Hero Victim. For 
у: “They caught this man and 
cem like punishment, since 
Пу a form of punishment, 


punishmen 


anged js usua. 
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but since we do not know explicitly from the 
story that this man is being punished for an 
aggressive act, and since it is explicit that he is 
being aggressed against, we should put a score 
of 10 in the Hero Victim category. 


No punishment of any kind. | 

Feelings of guilt, sorrow, depression, re- 
gret, or bad conscience. (Score only if ex- 
plicit, as, e.g., "He feels sorry he did it,” "He 
feels bad,” etc.) 

4 Nonphysical punishment from the environ- 
ment: deprivation, rejection by others, loss 
of loved persons or objects. 

6 Physical punishment: bodily harm, mis- 
fortune, caught by police, imprisonment. 

8 Physical punishment resulting in death: 

execution, Suicide when it is self-punishment 


for an act of aggression. (Also scored under 
Suicide.) 


bo 


Defenses 


Defense is scored only when the Aggression 
score is 6 or greater, It represents a defense 
against aggression-anxiety. Defense is never 
scored unless one of the following is true in 
connection with the scored aggression. (The De- 


fense score applies to Suicide as well as to 
Aggression.) 


4. Storyteller rejects an act 
disapproval of it. 

b. Storyteller denies something, ("That couldn't 
be a gun." "She doesn't want to hurt her.") 

c. The aggressive activity is excused or ration- 
alized. Aggression is put in a socially accept- 
able form (accident, self-defense, Sports activity, 
warranted punishment, fighting or killing in 
war, police defending others, killing in the 
line of duty, etc.). 

d. The aggressive activity is a dream 
wish, fantasy, or plan of the story character, 
but it is not completed through action, 

е. The action is described as past or future, 
(This refers to the distant past or future; that 
is, the action does not take place within the 
time-span of the particular story.) 

f. The activity is qualified; maybe, perhaps, 
might be, probably, as if, doubt about the 
nature of the activity, or a nonaggressive 
alternative to the activity is suggested, ("This 
woman is either choking the other one or 
she's helping her because she's Sick.") 

g. Noncompletion of aggression planned by the 
fantasy character. ("He goes out to kill this 
man, but then he changes his mind." 

h. Displacement of aggression on to nonhuman 
objects. ("Man beats his dog." "Man kicks a 
chair, smashes a vase.") 

i. Cause of death is unknown, but it is not 
clearly due to natural causes, When there is 


of a figure; voices 


‚ thought, 


death in a story and it is not explicitly the 
result of aggression or of natural or acci- 
dental causes—in short, when there is no cvi- 
dence in the story for inferring the cause of 
death—score Aggression 10 and Defense i. 
("The man enters the room and finds a dead 
body on the floor.") = 


Other Variables 
The following are scored simply on 
the basis of their presence in a story. 


Natural Death. A character dies, or has just 
died, from presumably natural causes. 2. 

Suicide. A character takes his own life, or is 
in the act of doing so. 

Tabooed Violence. Scored in addition to ди 
Aggression score of 8, 9, or 10 when the ag 
sion is especially brutal, bloody, gory, repul- 
sive, primitive, or sadistic. Also, mutilation of the 
victim, nai: 

Sex. Heterosexual petting or intercourse. £ 1 
ting is scored as Sex only if it involves physica 
contact with the breasts or genitalia or explicitly 
arouses sexual impulses. ке 

Score Sex only if it is actually part of th 
Story and not merely suggestive. The language 
used is not important, А 

Tabooed Sex. Tabooed Sex refers to апу ©Х- 
plicit sexual activity which does not come under 
the heading of Sex, i.e., any sexual activity othe 
than heterosexual petting or intercourse. ^ 
amples that occur in these storie: homosexuality, 
incest, masturbation, bestiality, necrophilia’ 
fellatio, cunnilingus, and other perversions. 
the stories these technical terms are not uset 
It is assumed that the scorer knows the slang 
terms. 

Tabooed Language. Most of the tabooed lang: 
uage is in reference to sex. Do not score at 
swearing as Tabooed Language. Only words ш 
are generally considered “dirty” are 4 
taboo. Usually these are various synonyms ‘0 
Sexual intercourse, masturbation, sexual anatomy» 
and the like. 


Relationships between the various 
TAT variables were determined bY 
dichotomizing all distributions of guns 
for each variable as close to the median 
as possible and computing the contin- 
§ency coefficient as the measure of rel 
tionship (Table 1). 


Reliability of Scoring 


А е 
The Ss’ names were removed from th 
Stories, which were given code numbers 
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TABLE т 
CONTINGENCY COEFFICIENTS BETWEEN VARIABLES 
Variable ua |: 2 3 4 5 6 7 8 9 то 
1. Ацигезвзїоп.........- | .44| -34*| -25 |—-12 14 88°] -22 4% .16 
2. Punishment....... ++ | .05 17 17 .32% 19 17 |— .08 .38ї 
3. Defense... se emn | -—.27 .22 21 |—.11 |—.12 oo 17 
LL EIE —.ast| ast] 371-461 -661| это 
5. Natural Deaths 2 n os |—.16 |—.12 |—.24 o3 
6. Suicide... o eee +15 .00 .o6 .04 
7. Tabooed Violence... . .38 .a71| -16 
8. Tabooed Sex. ....--- sit- 19 
о. Tabooed Language... | -09 
то. Number of Words... - 


* Significant at or below the .os level. 
f Significant at or below the .от level. 


so that the experimenter did not know 
while scoring the stories to which group 
the Ss belonged. Another person, work- 
ing independently without any knowl- 
edge of the purpose of the study or the 
nature of the Ss, scored all the odd- 
numbered protocols. The percentage of 
agreement in scoring the various TAT 
categories ranged from 86% to 99%; 
agreement for all categories combined 


was 95%- 


RESULTS 


Aggressive Fantasy Content 


The distribution of Total Aggression 
Scores for the combined groups was 
dichotomized at the median and the 
significance of differences between the 
three groups of Ss was tested by means 
of the chi-square statistic. Only when 
the differences between all three groups 
reached the .05 level of significance were 
differences between pairs of groups tested 
for significance. (То facilitate direct com- 
parisons of the frequencies in the three 
groups, all Ns in Tables 2, $› and 4 have 


been made directly comparable. In order 
not to create the distorted impression of 
large differences that would be given if 
percentages were used, these have been 
multiplied by .25, so that all frequencies 
in Tables 2, 3, and 4 may be viewed 
as if there were 25 Ss in each group. The 
chi squares were, of course, computed 
from the original frequencies.) 


From Table 2-A it may be seen that both 
Hypotheses т and 2 were not substantiated, The 
Aggressive-Bad and Passive groups did not differ 
significantly in total TAT Aggression. The Ag- 
gressive-Good group had significantly (.05 level) 
less TAT aggression than the Aggressive-Bad 
group, but did not differ significantly from the 
Passive group. 

The relationships between the groups are 
essentially the same when comparisons are made 
of Aggression scores derived only from stories 
in which the Hero is the aggressor (Table 2-B). 

Hypothesis 3 was not substantiated. The 
groups did not differ significantly in the propor- 
tion of stories containing Mild Aggression 
(scores 2-6) to stories containing Strong Aggres- 
sion (score 8-10). Dichotomizing at the propor- 
tion of .go maximizes the differences between 
the groups and yields the largest chi square of 
all possible dichotomies. Yet the differences be- 
tween the groups did not attain significance 


(Table 2-C). 
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TABLE 2 
AGGRESSIVE FANTASY CONTENT 


Number of Subjects” 


| Chi Square? 


Ang. 


Agg.- 
i | | | Chi | Good 
tt | Ba 
TAT Variable bp " ies | Square | Bad | 
^ad | Good | Passive : 
Above Median 16 | 7 14 - | a | ats 
А ssion Score | 2 4.42 | 
HU Таа Agere Below Median | o 18 T NE 
Above Median | ı7 | п | 3 
B) Hero Aggressor Score | 5.18 | 
VB) Hie dá Below Median | 8 16 MET EX » 
Mild Age <.30° 6 2 6 | 
(C) Proportion of —| 5.20 
Strong Арда 2.29 3 10 Sl ug P. З 
Hero Victim >40 | 8 ° 8 f 
(D) Proportion of .26 | 
Hero Aggressor <.50 7 6 7 D 2 к 
т ог more stories, 15 16 | 8 | | 
(Е) Suicide 74 | 
попе 10 9 | tg | И 


t With Yates’ correction, 
Though the Ns of the three groups were actu: 
of 25 in each group to facilitate direct сотрагіхо! 
© Aggression scores 2-6, 
Aggression scores 8-10. 


f This cut gives the largest chi square of all possible cuts, 
? «os. 


Hypothesis 4 was not substantiated. The 
groups did not differ significantly in the propor- 
tion of stories in which the Hero was the victim 
of aggression to stories in which the Hero 
was the aggressor, Dichotomizing at the propor- 
tion of .50 yields the largest chi square of all 
possible dichotomies, yet it falls far short of 
significance (Table 2-D). 

Hypothesis 5 was not substantiated. The 
groups did not differ significantly in the nu 
of stories containing the theme 
Dichotomizing simply on the basis of presence or 
absence of Suicide in the entire set of 10 stories 
comes closest to dividing the distribution of 
Suicide scores at the median; no other d 
yields a larger chi square for 
(Table 2-E). 

Hypothesis 6 was not substantiated, The three 
groups did not differ significantly in the propor- 
tion of Punishment relative to Aggression in 
their TAT stories (Table 3-A). When the groups 
are compared for themes of Punishment, not on 
a proportional basis, but on the basis of ab. 

solute number of Punishment themes in their 
stories, the differences between the groups are 
even smaller. 

Hypothesis 7 was not substantiated. The three 
groups did not differ significantly in the propor- 
tion of Defense relative to Aggression in their 
TAT stories (Table 3-B). 

Hypothesis 8. Since the findings with regard 
to Hypotheses 6 and 7, while not statistically 
significant, were in the predicted direction, and 


mber 
of Suicide. 


ichotomy 
these data 


“equalized” to an № 
ally 25, 22, and 27 respectively, they have here been “equalized” to 
ns. The chi squares are based 


on the actual Ns. 


since both Punishment and Defense arc r 
chologically similar in that they may be 2i his 
preted as indicative of the S's disapproval о тете 
aggressive fantasy, Punishment and Defense pe" 
regarded as equivalent in Hypothesis 8. T ie 
be argued that if ап 5 uses one means ч" not 
approving” his aggressive fantasy, he need Ed 
use another means in the same story. The ee 
were therefore compared on the basis сугара 
Proportion of either Punishment or ри" in 
relative to aggression, Themes of aggression ase 
Which there is neither Punishment nor merens 
may be called “raw” or unmodified aggre 
fantasy, When the distribution of raw ABRICT 
Scores is dichotomized as near the E UM 
Possible, it is seen that the groups арды 
nificantly (T; ve-Bad 8^ 
had ion in their stories ps. 
e-Good or Passive Less 
Which did not differ significantly {гөй "à 
other, Thus, Hypothesis 8 was substantiate. ted. 
Hypothesis 9 was only partially substan e. y 
The Aggressive-Good group had significa the 
more themes of Natural Death than GE not 
Aggressive-Bad Or Passive groups, which did 
differ significantly from each other (Table 3 


Behavior-Sample Aspects 


Ay pothesis то was amply substantiated 
All the beh a 
criminated 
:01 to 


avior-sample variables die 
between the groups at i. 
‘001 level of significance (1 able 4 


— A .ا‎ 0. 


-— 


a 
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TABLE 3 
MODIFIERS OF AGGRESSIVE FANTASY 


| | Number of Subjects | Chi Square? 
TAT Variable | Modifier ne Kex. | уь Chi Square | Agg.-Bad | Agg.-Bad | Agg.-Good 
ARR AES. assive v M 
Е _ | | Bad Good | | Agg.-Good| Passive | Passive 
| With Punishment? о | m 7 | 
(А) Aggression? | — — | х 2.05 
_ | No Punishment 15 п | 16 | 
| With Defense? 13 16 18 
(B) Aggression? ID мави а | 2.64 
No Defense m e | | 
| With Punishment | | | | | 
~ or Defense! | 4 15 12 
(C) Aggression? - ا‎ | ro. 7.43" 45# | 22 
No Punishment E | | 
or Defense 16 Z 9 
Present? 6 17 8 
(D) Natural Death - | اس‎ | 10.44" 7.54" 72 m 
None xo | 8 17 | | 
ù With Yates’ correction. 
Sce footnote b in Table 2. r " 
Aggression, thus comparing the groups on the basis of the pro- 


ed only on Ss having one or more tories containing 
of Punishment relative to Aggression. 
ased only on Ss having Ao or more tories conta 
proportion of Punishment or Defense relative to Ager 
* In one or more stories. 
In two or more stories. 
т р<.оз. 
p «ot. 


ion. 


Because of the markedly skewed distribu- 
tion of scores on each of these variables, 
the group comparisons were made on the 
basis of the number of Ss in each group 
having a given number of stories con- 
taining the particular TAT variable. In 
some cases a true picture of the differ- 
ences between the groups is obtained only 
when the distribution of scores is di- 
chotomized at more than one point. This 
was the case with Sex and Tabooed Sex. 


On every variable in able 4 the Aggressive- 
Bad group was significantly ahead of the Aggres- 
-Good and Passive groups, which did not 
differ significantly from cach other. The one 
partial exception to this, shown in Table 4-A, 
is Sex in one or more stories, Here the Aggres- 
sive-Bad and Aggre ve-Good groups did not 
differ. significantly from cach other, but both 
differed significantly (01 to .001 level) from the 
Passive group, which had very few themes of 
Sex. However, when the groups are compared on 
the basis of the number of Ss having two or more 
Stories containing Sex themes, the Aggressive- 
Bad group is far in the lead, while the Aggres- 
sive-Good and Passive groups do not differ 


significantly. The reason for this becomes obvious 
When the groups are compared on the individual 
ntly afraid 


TAT cards, The Passive Ss, appare! 1 
of doing anything that they think might incur 


ining Aggression, thus comparing the groups on the basis of the 


disapproval, did not respond with sexual themes 
even to Card 13MF, which usually elicits sexual 
themes. Indeed, sexual themes may be regarded 
as appropriate for this picture of a man standing 
beside a half-nude woman lying on a bed. The 
majority of sexual themes in the Aggressive-Good 
group were given to this picture. The Aggressive- 
Bad group, on the other hand, not only produced 
more stories with Sex, but distributed the sexual 
themes through all ten stories. Therefore, the 
result shown in Table 4-A for two or more 
stories is in effect what would be obtained by 
removing Card 13MF from the series, From this 
it may be concluded that the presence of sexual 
themes in stories produced in response to pic- 
tures not directly suggestive of sexual themes is 
associated with socially unacceptable overt ag- 
gression, The absence of sexual themes in stories 
produced in response to sexually suggestive pic- 
tures is associated with the Passive type of overt 
behavior. 

Table 4-B, C, D, E shows the marked differ- 
ences between the groups on the tabooed themes, 
the Aggressive-Bad group always far in the lead, 
with the Aggressive-Good and Passive groups 
showing no significant differences between them. 
Tabooed material in TAT stories is clearly 
associated with socially unacceptable overt ag- 
gressive behavior. 


DISCUSSION AND CONCLUSIONS 


Responses to a projective test may be 
thought of as consisting essentially of 
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TABLE 4 
BEHAVIOR-SAMPLE ASPECTS 


ARTHUR R. JENSEN | 


Number of Subjects? Chi Square? 
i Number of Chi Square | Agg.-Bad 
TAT Variable Stories Ages | Age Пък q zol 
Bad Good Agg.-Good 
r mı r 17 6 | b 
т or тоге ab 
= x 15.439 .07 11.09° 7.13 
попе 6 8 19 = 
(A) Sex p 
2 ог more 14 4 ^ 
m.at 4.09* 7.99* 12 
т ог none її 19 21 al] 
1 ог more 7 2 2 | | 
| 5.23 
попе 18 23 23 
(В) Tabooed Sex 
2 or more sA o o 
14.92% 10.389 | 6.50" же | 
т or попе 18 25 25 
т ог тоге 9 1 3 
(C) Tabooed Language 9.18 5.16" 3.24 “89 
попе 16 24 22 
1 ог тоге 10 о о " 5 
(D) Tabooed Violence 22.419 8.020 10.92 e 
none 15 25 25 — 
۶ I or more 13 3 3 
(E) Combined Taboo 13.818 6.06" 8.360 © 
(B, C, D) none 12 22 22 


two kinds of elements: those which are 
fantasy projections of inner tensions 
(drive states or conflicts), and those which 
are samples of the person's behavior. 
These two elements may be referred to 
respectively as the "projective" and the 
"behavior- (or trait) sample" aspects of 
the test response. The thematic content 
of TAT stories carries the largest portion 
of the projective element, while the so- 
called formal aspects (S's approach, lan- 
guage, style, perceptual and cognitive use 
of the stimulus, attitudes toward the task, 
etc) represent the behavior- or trait- 
sampling element. 


The results of this study do not support the 
projective hypothesis that n Aggression in TAT 
fantasy is related to overt aggressiveness as a 
general disposition or trait. There appeared to 
be neither a direct nor an inverse relationship 
between fantasy aggression and overt behavior. 
As pointed out previously, the direct-relationship 
hypothesis has found support only in studies 
dealing with temporary affective states or drive 
states (3, 6, 12, 16, 17). Thus the TAT, as re- 


gards its fantasy element, seems to be more ы 
test of the S's current mood and situational ot 
titude, and possibly also of certain persisten 
underlying conflicts or needs, rather than а (6 
of overt personality, the 
The “dynamic hypothesis” is based on 1- 
idea that overt behavior is a resultant of a ba t 
ance of inner forces and that inferences je 
overt behavior from fantasy content must s 
into account not only the various dE aul 
"needs" and "presses" represented in fantasy, ne 
also their interaction with each other. Thus p» 
degree of overt aggressiveness may be ene 
a resultant of an aggressive impulse айй ol 
anxiety that opposes aggression. The haline nit 
these forces should, according to the dyes 
hypothesis, provide the best indicator of ! 
degree of overt aggression. i 
This hypothesis found some support in An 
Present study, as it has in others (14 15) st | 
absence of punishment or of defense аа 
aggression anxiety seems to be associated hen | 
socially unacceptable overt aggression. The рї ms 
ence of natural death in the stories also Pdf 
to serve a defensive purpose and is posi ee 


correlated. with socially acceptable overt agi 
siveness, 


the 


$ " hat 

The behavior-sample hypothesis requires th : 

evidence of the 5% behavioral traits be T 
in his test Performance. The 5% response (0 
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test situation is regarded as a sample of his mode 
of responding to other tasks and interpersonal 
relationships. Examples of this in the clinical use 
of the ТАТ (and Rorschach) are the noting 
of signs of confusion, bizarre verbalizations, per- 
ceptual distortions, etc. as signs of schizophrenia; 
hesitancy, indecision, blocking, etc. as signs of 
anxiety or obsessiveness; and so оп. These are 
samples of the S's behavior. 

It was hypothesized in this study that the 
Aggressive-Bad Ss would include in their TAT 
stories defiant, antisocial, or "shocking" elements 
characteristic of their contempt for school- 
behavior propriety, and would manifest their 
lack of inhibition about expressing themselves 
in socially disapproved ways by giving freer 
expression to any socially tabooed themes that 
the pictures might suggest, however remotely. 
The evidence strongly supported this hypothesis. 
On each of the “tabooed” themes (Sex, Tabooed 
Sex, Tabooed Language, Tabooed Violence) the 
Aggressive-Bad group was far in the lead. This 
certainly cannot be interpreted as representing 
gross differences in the strength of the sex drive 
(п Sex) in the three groups. The differences be- 
tween the groups were consistent in direction on 
all pictures for all the tabooed variables. This 
Was not the case with any of the other variables 
in the study, Whereas certain pictures elicited a 
particular kind of thematic content more than 
others, the behavior-sample variables, such as 
Tabooed Language, were elicited about equally 
by all the pictures. 

The obvious conclusion is that the behavior- 
sample elements of the TAT response, when 
elicited under the testing conditions described 
in this study, are much more highly related to 
overt behavior than the projective or thematic 
clements, The reason is simple: the formal and 
trait-sample elements are the overt behavior. An 
analysis of the TAT stories from the behavior- 
sample point of view reveals more about the 
S's overt personality than an analysis of the 
thematic content per se. 

It is of course impossible to say with confi- 
dence what the results of this study would be 
if the Ss were girls, or were of a different age 
Broup, or were institutionalized delinquents, or 
mental patients. Also the method of administer- 
ing the TAT was undoubtedly of significance in 
the results; The number of sex and tabooed 
themes would have been much fewer, if they 
would have been expressed at all, had the test 
been individually administered. When the Sy- 
monds Picture-Story Test (which originally en 
cluded а picture of two nudes Солета 
Standing face-to-face) WS administer vec 
Yidually to зо high school boys, they б ped 
Tespon илә tabooed sex OF tabo 
an Ses containing tab 

Buage (19). 


s 


It should be pointed out that the three experi- 
mental groups were not conceived of as lying 
along a single continuum with respect to overt 
aggression. The main requirement was that the 
three groups differ markedly with respect to 
overt aggressive behavior. This was achieved. 
At least two dimensions would probably be 
required to account for the behavioral differ- 
ences among the groups. In retrospect one might 
hypothesize as the two dimensions: "conformity- 
nonconformity" and  "activity-passivity." The 
Aggressive-Bad Ss might be characterized as 
“actively nonconformist,” the Aggressive-Good 
Ss as “actively conformist,” the Passive Ss as 
“passively conformist." 

Another point: the selection of the 10 TAT 
pictures could possibly affect the outcome of 
such an experiment. The fact that the various 
TAT cards have different stimulus values is 
well-known, Certain pictures elicit more of a 
particular theme than do others, A full presenta- 
tion of the data of this study (9) clearly reveals 
that, in addition to this fact, the proportions in 
which certain themes are elicited by various 
pictures are not the same for different types of 
Ss. For example, Card 17GF elicited the most 
fantasy aggression from the Passive group and 
the least from the Aggressive-Bad group, while 
Card 20 was just the reverse. The implication 
would seem to be that the same variable may 
carry a different. weight when elicited by one 
picture than by another. Aggression elicited by 
Card 17СЕ was negatively correlated with overt 
aggressiveness, while aggression elicited by Card 
was positively correlated with overt aggres- 
;s. Future research might apply an "item 
' technique. to the TAT, treating the 
individual cards as items and correlating the 
themes elicited by each one with various per- 
sonality characteristics. It is not improbable that 
such an approach to TAT analysis could have 
greater predictive value than the more global. 
or additive approaches, in which the differential 
stimulus values of the various pictures are not 
so systematically taken into account. 


Finally, an observation that is made 
too seldom in studies such as this, is the 
fact that, even though we were here 
dealing with groups differing in the ex- 
treme, the TAT variables discriminated 
very poorly for practical purposes. The 
statistically significant discriminations 
must be regarded as of only theoretical 
interest. Using any combination of these 
statistically significant variables for pre- 
dictive purposes, even with extreme 


groups, would obviously result in a very 
large percentage of errors, as may be 
seen from Tables 2, 3, and 4. That the 
TAT can be used for individual predic- 
tion of overt behavior is thus seriously 
questioned by these results, especially 
as regards the use of fantasy "needs" 
and "presses." Somewhat more confidence 
may be had in behavioral predictions 
from the TAT when the S's performance 
is interpreted as a behavior-sample. 


SUMMARY 


The relationship of Thematic Ap- 
perception Test responses to overt be- 
havior was investigated with respect to 
aggression. The central question was: 
How is behavioral aggressiveness re- 
flected in the TAT? The major hypoth- 
еѕеѕ were: (a) There is a direct rela- 
tionship between aggression in ТАТ 
fantasy and in overt behavior. (b) There 
is an inverse relationship between the 
amount of fantasied punishment, rela- 
tive to fantasied aggression, and overt 
aggressive behavior. (c) There is an in- 
verse relationship between the amount of 
defense against aggression anxiety in the 
fantasy and overt aggressive behavior. 
(d) There is a direct relationship between 
those aspects of the T AT response which 
may be viewed as behavior-samples and 
overt behavior in social situations. With 
respect to socially disapproved overt 
aggressiveness, the TAT responses would 
be expected to evince samples of socially 
disapproved, tabooed, or defiant be- 
havior. 

The method of the study was to permit 
whatever relationships that might exist 
between the TAT and the behavioral 
variables the maximum opportunity for 
manifesting themselves, by comparing 
relatively homogeneous groups of Ss dif- 
fering widely in their overt behavior 
with respect to aggressiveness. From an 
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all-boy high school three groups of Ss 
were selected by means of teacher ratings: 
(a) 25 who were aggressive in socially 
unacceptable ways (Aggressive-Dad); (b) 
22 who were aggressive in socially ac- 
ceptable ways (Aggressive-Good); (0) 27 
who showed no overt aggressiveness 
(Passive). The Ss wrote stories to 10 TAT 
cards selected for their tendency to elicit 
themes of aggression. The stories were 
scored on the following variables: Ав- 
gression (Hero Aggressor, Hero Victim, 
Indeterminate), Punishment, Defense 
(against aggression anxiety), Suicide, 
Natural Death, Sex, Tabooed Sex, Ta 
booed Violence, and Tabooed Language. 
Interrater agreement was determined 
and found to be high (95%0). А 

The “direct relationship” hypothesis 
was not substantiated, The Aggressive 
Bad and Passive 55 had about the вате 
amount of aggression in their Л Al 
fantasies. The Aggressive-Good group 
had even less fantasy aggression than the 
Passive group. The groups did not differ 
in the proportion of aggressive stories 10 
which the Hero was the aggressor or the 
victim of the aggression. Nor did e 
groups differ in the amount of mild. o1 
socially acceptable forms of aggression 
in their stories. 

The three groups differed at the e 
level of significance in the proportion 0 
aggressive stories which contained neither 
Punishment nor Defense. The Aggre 
sive-Bad group had a greater proportion 
of such unmodified or "raw" aggression 

The theme Natural Death was Ду, 
pothesized as being a defense agains 
aggression anxiety, representing ап aa 
gressive wish in disguise. The Aggress" ii 
Good group had significantly C 
Natural Death in their stories than t? 
other groups. А 

The groups did not differ significantly 
in the number of Suicide themes. 


PN 


= J < a 
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The behavior-sample aspects of the 
TAT stories showed highly significant 
differences between the groups. All the 
differences on the following variables 
were significant at the .o1 to .o01 level of 
confidence, with the Aggressive-Bad 
group far in the lead and the Aggressive- 
Good and Passive differing from cach 
other hardly at all: Sex, Tabooed Sex, 
Tabooed Language, and Tabooed Vio- 
lence. This relationship held up for these 
variables on all ten TAT pictures; this 
was not the case with any of the other 
variables in the study. 

The following general conclusions 


were drawn: (а) There was very little, 
if any, relationship between aggression 
in fantasy and in overt behavior. (a) The 
absence of themes of punishment and of 
defenses against aggression anxiety in 
the TAT was associated with socially 
unacceptable forms of overt aggression. 
(c) Aspects of the TAT responses which 
were regarded as behavior samples were 
related to overt aggressive behavior at a 
high level of significance. Ss who habitu- 
ally acted-out aggressively in ways re- 
garded as taboo in the school setting 
responded also to the TAT with socially 
tabooed content and language. 


REFERENCES 


1. ATKINSON, J. W., & MCCLELLAND, D. C. The 
projective expression of needs; П. The 
effect of different intensities of the hunger 
drive on thematic apperception. J. exp. 
Psychol., 1948, 38, 643-658. 

. BALKEN, E. R., & MASSERMAN, p H. The 
language of fantasy: HI. The language of 
the fantasies of patients with conversion 
hysteria, anxiety state, and obsessive-com- 
pulsive neurosis. J. Psychol., 1940, 10, 75-86. 

3. BELLAK, L. The concept of projection: ап 

experimental investigation and study of 
the concept. Psychiatry, 1044, 7, 353-370. 
4. CLARK, R. А. The projective measurement of 
experimentally induced levels of sexual 
motivation, J. exp. Psychol, 1952, 44 
391-399. 
. COHEN, А. К. Delinquent boys: The culture 
of the gang. Glencoe, Ill.: The Free Press, 
1955. А 
6. Frsunacu, S. The drive-reducing function of 
fantasy behavior. J. abnorm. soc. Psychol., 
1058, 50, 31! à P 

ГА meu и n. The relationship Deier 
hostility in the TAT and d 
hostility. J. proj. Tech, 1055, 19, > ai 
8. Hartman, А. A. An experimental exan 
nation of the thematic арры al 
technique in clinical dag a REA NG: 
Monogr., 1919, 63, No. 8 (Wh А 
303). 8 í asy and 

о. Jensen, A. R. Aggression ртр 
overt behavior. Unpublishe К шы 
sertation, Teachers College, C9 
Univer., 1956. 

10, KonNER, ANNELIESE F. Theore 

tions concerning the scope anc 


ю 


х 


tical considera- 
1 limitations 


(Accepted for 


publica tion 


of projective techniques. J. abnorm. soc. 
Psychol., 1950, 45, 619-627. 

11. Linpzey, С. An experimental examination of 
the scapegoat theory of prejudice. J. 
abnorm. soc. Psychol, 1950, 45, 296-309. 

12. MCCLELLAND, D. C., ATKINSON, J. W., CLARK, 
R. A, & LOWELL, Е. L. The achievement 
motive. New York: Appleton-Century- 
Crofts, 1953. 

13. Murray, Н. А. Thematic Apperception Test 
manual, Cambridge: Harvard Univer. Press, 
1943. 

14. Mussen, Р. Н., & NAYLOR, Н. K. The re- 
lationship between overt and fantasy ag- 
gression. J. abnorm. soc. Psychol., 1954, 
49, 235-240. E 

15. PrrTLUCK, PATRICIA, The relation between 
aggressive fantasy and overt behavior, Un- 
published doctoral dissertation, Yale Uni- 
ver., 1950. 

16. SANFORD, R. N. The effects of abstinence 
from food upon imaginal processes: a 
preliminary experiment. J. Psychol., 1936, 
2, 129-136. 

17. SANFORD, R. №. The effects of abstinence 
from food upon imaginal processes: а 
further experiment. J. Psychol, 1937, 3, 
145-159. 

18. SANFORD, R. ADKINS, M. M., MILLER, R. B., 
Cons, E. A., & others. Physique, person- 
ality and scholarship: a cooperative study 
of school children. Monogr. Soc. Res. Child 
Developm., 1943. 8, No. 1 (Serial No. 34). 

1g. SYMONDS, P. M. Adolescent fantasy: an in- 

k vestigation of the picture story method of 
personality study. New York: Columbia 


Univer. Press, 1949- 


March 11, 1957) 
| Bureau 
{МЮ м 


| Dated 


| 


ite 


Edni.ccy. Research 
COLLEGE 


 -— 


—— TU WM Rn 


Vol. 71, No. 17 


Whole No. 446, 1957 


Psychological Monographs: General and Applied 


A Pattern Analysis of Descriptions of "Best" and ‘Poorest’ 
Mechanics Compared. with Factor-Analytic Results' 


Louis L. McQuitry* 
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Т particular method of pattern 
analysis developed in this paper is 
called agreement analysis. It can be in- 
troduced by contrasting it with factor 
analysis. These two methods differ in 
terms of the assumptions they imply con- 
cerning the interrelationships in subject 
matter they are designed to analyze. 
Agreement analysis is based on a gen- 
eralized theory of types. The types are 
assumed to reflect themselves in many 
kinds of interrelationships among char- 
acteristics—not just linear ones. Conse- 
quently, agreement analysis is a tool for 
determining all of the various ways in 
which characteristics are presumed to 
concatenate to yield types. Having this 
general ability to isolate types in terms of 
all kinds of interrelationships, it is, of 
course, capable of isolating them in terms 
of any restricted kind of interrelation- 
ships among characteristics. 


Factor analysis, on the other hand, 
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assumes that characteristics concatenate 
in terms of linear interrelationships only. 
It is a tool for analyzing the linear inter- 
relationships among characteristics and 
[ог expressing these in terms of factors. 
Types which may be discovered are 
limited either to standings on single fac- 
tors or configurations of standings on 
several factors. 


A more detailed contrast between factor analy- 
sis and agreement analysis will help give addi- 
tional meaning to the latter approach. The 
most common method of factor analysis (R- 
technique) has as its purpose to take the inter- 
correlations between a large number of indexes 
of personality and analyze these to yield a few 
fundamental personality traits in terms of which 
the indexes can be classified. 

An important consideration in this factor 
analytic. procedure is that the intercorrelations 
are taken across all subjects of a sample; no 
allowance is given to the possibility that variables 
might Ье interrelated differently in various 
categories of subjects of a single sample. If 
variables are interrelated differently among 
categories of subjects and if the number of sub- 
jects representing each category varies con- 
siderably, then intercorrelation across all sub- 
jects as generally taken in factor analysis would 
represent а hodgepodge. 

Agreement analysis assumes types and is a 
method designed to assist in their isolation; it at- 
tempts to isolate them by classifying together 
individuals who give identical responses to large 
numbers of items of a test. 

Substantiation of a need for a method which 
contrasts with factor analysis is simple. Factor 
analysis docs not prove the existence of factors; 
jt merely clarifies the characteristics which they 
have if we postulate them. 

A limiting characteristic of factor analysis is 
the fact that it does not include a statistic for 
rejecting it as not fitting data particularly well. 


One way to help correct for this to 
Se sak, earch 
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develop methods based on the assumption of 
constructs other than factors, and compare rc- 
sults obtained with the two methods. 

Agreement analysis postulates onc kind of 
construct (types) which contrasts with factors. 
Applications of agreement anal will help us 
to learn some of the characteristics which its 
types have. Comparisons of results from factor 
and agreement analyses will assist us in deter- 
mining which alternative postulate is more help- 
ful for various purposes. 


This paper includes four main sec- 
tions. The first sections develops the 
theory and method of agreement analy- 
sis and illustrates the method by applying 
it to descriptions of "best" and "poorest" 
mechanics. The second section compares 
the results of the agreement analysis with 
those of a factor analysis of common 
data, and interprets the types in the 
light of both the factor and agreement 
analyses. The third section critiques the 
method of agreement analysis. The 
fourth section summarizes the entire 
paper. 


I. CLASSIFICATION OF DESCRIPTIONS BY 
AGREEMENT ANALYSIS 

As mentioned above, an investigation 
by agreement analysis is based on a 
theory of types. In the particular theory 
of the present investigation, the concept 
of type is treated as an intrinsic con- 
struct, Each type is assumed to reflect 
itself in a pattern of characteristics. Con- 
sequently, individuals can be classified 
into typological categories according to 
the patterns of characteristics which they 
exhibit. Each category into which indi- 
viduals are classified is said to represent 
a type. The individuals of any one type 
ate presumed to have certain character- 
istics in common. These characteristics 
are called a. pattern. The individuals of 
anv other type are also presumed to 
exhibit a pattern of characteristics, and 
their pattern is unique to their type. In 
brief, the individuals of each type ex- 


+ 


hibit a pattern of common characteristics 
ample, 


which is peculiar to them. For е 
Туре X might include individuals who 
exhibit a pattern of dependability, carc- 
fulness, cooperativeness, and industrious 
ness; Type Y might include individuals 
who exhibit the pattern of carefreeness. 
friendliness, joviality, and slipshodness. 


The Linnaean approach (1, pp. 409-192! tp 
the classification of living organisms in the field 
of biology results-in a chart which shows (a) 
all types of living organisms and (b) the pat 
terns of characteristics peculiar to each type. M 
the lowest level of classification of living things: 
cach organism is classified into a species, Gon 
sequently, each species includes а number of 
organisms. The organisms of cach species CX 
hibit a number of common characteristics which 
are peculiar to them, 

At the next level of classifi 
is grouped with other spe 
Consequently, cach genus contains 
species. The species of each genus have cer 
characteristics in common which are peculiar to 
them. In an analogous fashion, genera are 
grouped to form families, families to form 
orders, and so on through classes, phyla, and 
kingdoms. 

\ simplified and incomplete Linnacan chart 
is shown in Fig. 1 for illustrative purposes. А! 
the bottom level of the chart, we have several 


individual organisms, then at the next level 
ete. 


ation, cach species 
s to form genera. 
several 
jin 


we have several species, then several gener. 
up to a c There is, of course, a pattern 
characteristics for cach species, genus, family. 
etc. 


of 


Analogously to the simplified Lin 
naean Chart, we propose to build а 
classification of certain. descriptions ш 
mechanics. 

Since so little is known about the type 
into which desc riptions of mechanics таў 
ultimately be classified, we propose от 
the present to use the Linnacan terms 
of species, genera, families, etc lor 
categorizing them. k 

We have six purposes in building неч 
classification; three of them are method- 
ological, and the other three are ШУ 
stantive. The methodological purpose 
are (a) to illustrate a quantitative metho 
for developing the classification, (?) 
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Fic. 1. A simplified and incomplete Linnaean chart. 


indicate some of the potentialities of the 
method in the field of personnel research, 
and (c) to compare results from this pat- 
tern analytic approach with а factor 
analysis of common data. 

The substantive purposes are to make 
a preliminary investigation of the follow- 
ing hypotheses: (a) that the descriptions 
of mechanics selected as "best" and 
“ 

poorest” сап be classified into types; 
(b) that the classifications are relatively 
dependable in the sense that two separate 
classifications of subjects on the basis of 
two sets of test items will yield similar 
results; and (c) that the classifications 
are valid in the sense that they are re- 
lated to an external criterion. 

The above purposes are, of course, related to 
ч theoretical, psychological position which 
should be specified unless it otherwise be mis- 
Understood, We are not attempting, initially, to 
isolate the types which may in some way be said 
to be fundamental for the generalized under- 
Standing of human behavior. For the time being. 
at least, it is assumed that different types may 
be Significant for different purposes. For ex- 
ample, types which derive from an analysis of 
Symptoms of diseases may be significant for 
differentiating between mentally and physically 
ill patients, but other types which derive from 
an analysis of characteristics of "best" and 
poorest" mechanics may be required in differ- 
entiating subjects in relation to their qualifica- 
lions as mechanics, We are attempting to isolate 
lypes which are significantly related to a cri- 


terion, but we are not arguing that these single 
criterion types are the most fundamental ones in 
terms of which to understand behavior in 
general. On the contrary, we are inclined to the 
point of view that behavior can be categorized 
in many different ways and that each way will 
give somewhat different types. 

Even though we are interested initially in 
types pertinent to single criteria, we are, never- 
theless, also oriented ultimately toward the more 
fundamental scientific problem of deriving a 
parsimonious, systematic structure for the in- 
clusive representation of behavioral organization. 
Types which would fulfill this requirement 
would be minimal in number, simple in descrip- 
tion, and of maximal value in predicting 
behavior. 

The single-criterion studies can assist in the 
solution of the more fundamental problem, The 
types which derive from the single-criterion 
studies can be used as the basis for the develop- 
ment of those which are more generally mean- 
ingful, and therefore more basic, scientifically. 
Each single-criterion study will have selected a 
set of items, significant for the criterion under 
consideration. All of the items taken jointly 
from several single-criterion studies and analyzed 
simultaneously in such a manner as to isolate 
all the types represented in the data should 
produce those more comprehensive types which 
are pertinent to all the scveral single criteria. 
Consequently, these types should be relatively 
fundamental in the sense that they would be 
generally applicable in the prediction of be- 
havior. 

Since there is probably no absolute limit to 
the number of single criteria which may be used, 
and since each additional one may add some- 
thing, it will probably always be possible to im- 
prove somewhat on the classification into funda- 
mental types, which derive from this approach. 
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However, the rate of improvement is probably 


asymptotic, or at least would become so at some 
point, so that eventually the amount of im- 
provement which could be added would be- 
come very small. 


The Method of Classification 


We are now prepared to outline and 
illustrate with data a method of agrce- 
ment analysis which we have developed 
for the isolation of types. It will here 
be outlined as it is applied in a single- 
criterion study, 

Subject and tests. Agreement analysis 
was applied to data which had been 
gathered in an earlier factor ralytic 
study by McQuitty, Wrigley, and Gaier 
(3). As pointed out in this earlier paper, 
an inventory of 200 items was developed 
using the language which mechanics used 
in describing other mechanics whom they 
had selected as representative of either 
"best," "poorest," or "average" mc- 
chanics. The inventory for describing 
mechanics was administered to 428 su- 
pervisors, who selected а 
“poorest,” or an “average” 


"best" a 
mechanic and 
described him by replying either "yes" 
or "no" to each of the 200 items of the 
inventory. In the present investigation, 
we used the results from supervisors who 
described either a "best" or a "poorest" 
mechanic; we did not use the results [rom 
those who described an "average" 
chanic. 

In the first. classification, we analyzed 
results from the first 120 items, In the 


me- 


second classification, we analyzed results 
from the remaining 80 items. The 200 
items of the inventory were listed in 
random order. The initial classification 
was based on the first 120 items because 
the factor analysis mentioned above was 
in terms of these items, and we wished 
to compare results from the two different 
kinds of analyses. 
describe the first classification in 


We shall 
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detail; the second one will be clear by analogy. 
For cach supervisor in the first classification, we 
20 responses which described а mechanic 
ed by him as either a "best" or a "poorest 

с. Each response is a description of 2 
mechanic, Each 120 responses by any one super 
visor is a pattern. of descriptions, Fach pattern 
of descriptions pertains to a particular mechanic, 
cither a "best" one or a "poorest" onc. " 

There were 112 patterns. pertaining to "best 
chanics and g2 pertaining to "poorest me 
chanics, giving а total of зор patterns, Using 
random numbers, we arbitrarily reduced the 112 
patterns for "best" mechanics to 108 to give 3 
grand total of patterns for “best” and “poorest 
mechanics, combined, of goo (for convenience 
of machine computations), 

Again using random numbers, we divided the 
108 patterns for "best" mechanics into equal: 
sized experimental and cross-validational n ples. 
We did likewise for the g2 patterns “шр 
of "poorest" mechanics, The two experimenta" 
samples, one for "best" and the other ier 
“poorest” mechanics, were used build OMe 
classification of patterns into type The Bros 
validational samples were used to determine A 
well the types held up on other samples of the 
same universe. 

As the patterns of descriptions are cl ; 
successively, into more and more inclusive tY 
some of the components of the patterns ES 
dropped. In other words, the initial patterns 
with which we start are reduced in their n 
bers of descriptions as the classification proces 
into more and more inclusive types, In order t 
differentiate these several “levels” of classification 
we shall speak of individual patterns ien 
classifying the initial ones into species; then а 
species patterns when we » classifying па 
senera: and of genus patterns when We aT 
classifying into families; and so on. 


m 


to 


assified, 
pes 
are 


Details of the method. The method of 


analysis here used is an abbreviated bid 
sion of a complete method of panen 
analysis, developed by the author ant 
called agreement analysis (6, 7) 398 
present version differs from the complete 
method in these respects: (а) it Ls 
calculation which corrects agreement 
scores for chance fluctuations: (2) it ap 
plies a somewhat less e acting, but aon 
rapid, technique for classifying patterns 
into the Categories which they resemble 
most; and (с) it limits each category 10 
not more than two patterns from he 
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next lower level of classification; it is 
therefore a bina ry version. 

This abbreviated version of agreement 
analysis is justified here because the pur- 
pose is not to isolate the most funda- 
Mental types but rather to study pat 
terns of responses in relation to an ex- 
ternal criterion. This version allows the 
investigator to isolate rather rapidly a 
great number of reasonably reliable pat- 
terns and to study them as predictors. 


Our abbreviated method of binary agreement 
analy involves a novel treatment of the agrec- 
Ment score developed by Zubin (10). ‘The agree- 
ment score is computed for pairs of individual 
patte . The agreement score for any two indi- 
vidual patterns is the number of identical de- 
scriptions contained in the two patterns. Suppose 
that the two individual patterns A and B both 
contain “yes” for the first five items of the 
inventory, disagree on the sixth item (one 
Pattern having a "yes" and the other pattern a 
no”), and that both patterns contain "no" for 
the next four items, On these 10 items, the two 
Patterns, A and B, would have an agreement 
Score of nine, because their answers would agree 
9n nine of the 10 items. 

Each individual pattern. was. paired with every 
other individual pattern, and an agreement score 
Was determined for each pair of individual pat- 
terns, These agreement scores are illustrated in 
йе 1 which shows 10 of the individual pat- 

descriptive of "best" mechanics. The 
patterns are labeled A through J and 
designated in the top row and left hand column 
of the table. 

The number, 102, in the intersection of row 
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A and column B of the table is the agreement 
score of the pair of individual patterns A and 
D. This same score also occurs in the intersection 
of row B and Column A. Other entries in Table 
1 are to be interpreted in an analogous fashion. 
Entries in the diagonal cells are omitted, since 
they would report the agreement score of cach 
individual pattern with itself and are not needed 
in the analysis. 

In the present analysis, Table 1 was expanded 
to include the agreement scores for all 54 in- 
dividual patterns descriptive of "best" mechanics 
in the experimental sample, i.c, the sample on 
ion of the descriptions of 
“best” mechanics was developed. Table 1 is shown 
as an abbreviated illustration of this larger 
matrix of agreement scores, 

The largest agreement score in the 54 X 54 
matrix was selected. Of all the individual pat- 
terns represented in the matrix, the two with 
the largest agreement score have more answers 
in common than any other two, These two pat- 
terns constitute. the first species. They and all 
agreement scores for cither of them are with- 
drawn from the matrix, The pair of patterns 
with the highest agreement score in the reduced 
matrix is next selected. This constitutes species 
number (wo. The remaining species are se- 
lected in an analogous fashion, 

In the case of a tie in agreement scores, such 
as between the individual patterns of pair K 
and L, on the one hand, with M and N, on the 
other, it is immaterial as to which species is with- 
n first. However, in the case of a tie such as 
between X and Y, on the one hand, with X and 
Z. on the other, it is material. In this latter kind 
of tie with one pattern common to two pairs, 
we have followed the practice of classifying X 
with Y rather than Z only if the next highest 
ement score in which Y occurs is lower than 
next highest in which Z occurs. By selecting 


which the cl 


draw: 


agre 
the 


TABLE r 


ILLUSTRATIVE Samre or “BesT” MECHANICS’ AGRE N INDIVIDUAL PATTERNS 
A B G D E H I J 

— -— = a =i 
A 102 до 93 93 99 03 от 02 m 
B 102 94 03 93 97 95 93 04 99 
c go 94 | 101 101 95 97 97 98 100 
D 93 93 10r 100 100 98 92 99 98 
E 03 03 тот тоо 102 96 02 os 04 
F 99 97 95 тоо 102 02 оо 97 86 
G 93 95 97 98 96 92 100 97 100 
н or 93 97 92 92 тоо тоз 3 
1 92 ba 98 оо 95 97 97 103 105 
J 99 99 IOI 08 94 96 100 тоо тоз 


а ; Зр с : Ag 
nd B is ro2; between individual patterns Aan 


ar ЫР 
id “agreement score," see text. 


NOTE Tr ў " read as follows: The agreement score between individual patterns A 
E.—The table is to be read d C is 9o; etc. For definition of "individual pattern" 
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Y instead of Z, the agreement score which will 
be used to classify Z is larger than would be the 
one to classify Y if we had reversed our choice. 
If the second highest agreement scores are also 
tied, reference can be made to the highest ones 
which are not tied. These can then be used as 
a basis for a decision. 

We have now outlined how to classify all 54 
individual patterns into species, each species con- 
taining two members, and only two. In those 
studies where there is one individual pattern 
left over, it is classified into an additional species 
containing only onc case. 

The classification of individual patterns into 
species started with a pattern of answers for 
cach mechanic described. The pattern of answers 
is the endorsements on the particular inventory 
used to describe the mechanic. The classification 
into species brings together those pairs of in- 
dividual patterns which are highly similar, as 
determined by agreement scores. We can usc 
this same approach for classifying species into 
genera, provided we can obtain a pattern of 
answers for cach species. 

A species' pattern of answers is those endorse- 
ments which are common to both individual 
patterns of the species. Suppose, for example, 
that both individual patterns X and Y of a 
species have "yes" for item 1 and "no" for item 
2. Their species pattern for the first two items 
would be "yes"."no," S pose further that in- 
dividual pattern X hasies" for item 8, while 
individual pattern Y has “no” for this item, and 
they both have “yes” for item 4. Their species 
pattern for the first four items would be "yes"- 
"no"-blank-"yes." The blank denotes the fact 
that the two individual patterns disagree on 
item 3. This item is dropped from consideration 
in the further classification of this species. In 
other species where both individual patterns 
have the same answer to the item, it is ret 

In the manner just described 
answers is determined for each Species which 
contains two individua] patterns, These are 
called species patterns. In those studies where 
the analysis started with an odd number of cases 
and ends in the last species containing but опе 
individual pattern, the pattern of answers for 
this species is the same as those of the one in- 
dividual pattern which the species includes, 


ained. 
‚ a pattern of 


Since we now have a pattern of an- 
swers for each species, we can use the 
same technique for classifying species 
patterns into genera as we originally 
used for classifying individual patterns 
into species. Further, we can use the 
same plan for determining the pattern 


of answers for each genus as we used 
for determining the pattern of answers 
for each species. The system is entirely 
repetitive at each successive level; we 
merely continue it for classifying species 
patterns into genera, genus patrerns Into 
families, etc., until the classification 15 
completed. B 

In an analogous fashion as outlined 
above, we classified the patterns descrip- 
tive of “poorest” mechanics. This classt- 
fication had to be discontinued at the 
class level, where there were only A 
classes, because the patterns for classes 
contained common responses to по еШ 

Unique features of the method. ue 
method of analysis contains a number ba 
unique features. It is essential to Р й 
preciate these if the implications of " 
method are to be understood and it ч 
sults from it are to be properly ng 
preted. These unique features can d 
introduced by showing how the memo 
is designed to capitalize on the pns 
possibilities of responses treated in CO" 
binations. Е 

Meehl (9) has illustrated that it es 
theoretically possible for рине 
treated in pairs to have relationships E 
à criterion even though the items treat A 
separately have zero relationship e 
criterion. This potentiality of pa sur 
treated in pairs would appear to ma " 
it highly desirable to analyze respons 
in combinations of two. However, = 
two-answer items taken two at à pe 
would yield 4.950 pairs of items pos 
19,800 Tesponse pairs for analysis. * 
analyze this many combinations a 
be an unreasonably laborious task. 5 
sides, to analyze in combinations of m 
is not an entirely satisfactory solutio", 
For, if pairs have potentialities over "e 
above Separate items, then by the sam 
logic triads have potentialities over А 
above pairs, and likewise any order of 


is 
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combination has potentialities over and 
above any one ol lower order. We are 
forced then to the conclusion that it is 
essential to treat items of a test in rela- 
üvely large combinations if we are to 
investigate thoroughly the predictive po- 
tentialities of the test. 

То analyze all items of a test in com- 
bination is not nearly as laborious as it 
would at first seem. Even though there 
ате many combinations of responses theo- 
retically possible, only a very few of them 
Occur empirically, and those which do 
Not occur need not concern us. 

In those patterns which do occur, some 
of the responses are relatively undepend- 
able and may be disregarded. A problem 
15 to determine which responses are rela- 
tively undependable. In our approach to 
this problem, we assume that a response 
тау be completely dependable for some 
Subjects and completely undependable 
for other subjects. 

This assignment of differential, all or 
none, dependability of responses in rela- 
tion to the subjects who give them is 
Novel in test analysis. The usual ap- 
Proach is to assign a single index of 
reliability for each item as computed 
from the average behavior with respect 
to it, even though some subjects may al- 
Ways give the same response to an item 
While others continually change in their 
Tesponses, 

_ Our approach, on the other hand, as- 
Signs a perfect reliability to an item for 
those subjects whose responses to it are 
Consistent with their responses to other 
items. For the same item, our approach 
assigns а zero reliability for those sub- 
jects whose responses to this item are 
inconsistent with their responses to other 
Items, 

In an effort to determine which responses аге 


consi а 
venient with other responses, we classify sub- 
Jects together as already explained in our method 


of agreement analysis. In this approach, some 
items are irrelevant (undependable) in the sense 
that they do not classify a pattern in the same 
fashion as the majority of the responses in a 
pattern, As an illustration, suppose that a test 
of 100 items is administered to 100 subjects, 
and that subjects A and B agree in the answers 
which they give to 99 of the 100 items. Suppose 
further that the next highest agreement score 
that either of these subjects has with any other 
subject is 80. We would conclude that А and B 
should be classified together in terms of their 
patterns of responses to the items of the test, 
and that their responses to the one item on 
which they disagree is irrelevant to their classi- 
fication. Their classification would be achieved in 
terms of their predominant patterns of responses, 
where these patterns are defined as the ones 
which include the greatest possible number of 
responses and still yield a classification. 

Consequently, any response which does not 
conform to the predominant pattern of responses 
is presumed to be irrelevant. It is assumed to 
vary independently of the responses in the pre- 
dominant pattern. In computing a correction for 
agreement scores in the complete method of 
analysis (6, 7) irrelevant responses are as- 
sumed to vary by chance with respect to the 
predominant patterns. 

A response which is irrelevant in the classifica- 
tion of one subject may, however, be relevant 
in the classification of another subject. Suppose 
that the one item on which subjects A and B 
of the above example disagree is item $; their 
responses to this item are irrelevant in classifying 
them together, Suppose further that C and D 
are classified together because they agree on 99 
(of 100 items) including item 2 but excluding 
item 3. In this case item 3 rather than item 2 
would be designated as irrelevant; item 2 would 
be relevant for subjects C and D but not for A 
and B. 

As the classification by agreement analysis 
proceeds from the species levels to higher ones, 
the criterion for judging responses to be relevant 
usually becomes more stringent, In order for a 
response to an item to be relevant in the first 
classification, a response to it has only to be 
given by both of two subjects who classify to- 
gether. As the classification proceeds, more and 
more individuals are classified into fewer but 
larger categories and all of the subjects of any 
one category have to agree on a response in 
order for it to be relevant to their classification. 

Since irrelevant responses are assumed to be 
independent of the predominant patterns which 
determine the classification, they are presumed 
to occur by chance with respect to the pre- 
dominant ones. Consequently, irrelevant re- 
sponses can be expected to yield both agree- 
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and disagreements between. predominant 
We can spot those which yield dis- 
agreements аз shown above. From these, we can 
estimate the number of irrelevant responses 
which yield agreements in any classification. For 
example, if subjects A and B who agreed on 99 
out of 100 items of a test were responding to 
items which have only two possible answers, we 
would expect that their agrcement on irrelevant 
responses would equal their disagreement on 
irrelevant responses, Since they disagreed on one 
irrelevant item, we would estimate that they 
agreed by chance on one irrelevant item and 
we would further estimate their true agreement 
score to'be 98 rather than оо. This example 
illustrates the logic on which our corrected 
agreement score is based for the complete ver- 
sion of agreement analysis (6, 7). 

It is helpful to emphasize the assumption on 
which the correction is based in order that 
agreement analysis can be properly understood. 
The correction docs not assume that cach answer 
alternative is equally apt to occur, as many col- 
leagues have originally thought when they first 
examined the method, It assumes instead that 
subjects are equally apt to give any one of the 
possible answer alternatives to those particular 


ments 
patterns. 
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Results from the Classification 

A portion of the results from classily- 
ing descriptions of "best" mechanics n 
shown in Table 2; space does not permit 
showing all of them. In the table, the 
lowest level of classification represents 
the individual patterns; the next level 
represents species; and then a genus 15 
shown. This table corresponds to only 
a small portion of the chart shown as 
our model in Fig. 1. It corresponds 10 
any section which includes one genus 
two species, and four individual pat- 
terns. 


Table 2 shows certain detailed information 
not shown in Fig. 1. The capital letters of К able 
are codes for patterns descriptive of АШ, 
mechanics, Thus the entry оп the left side a 
the table at the specics level means that ae 
species includes individual patterns К and S 
Under this species is a twice underlined number 


> E Р xe ^ dich in- 
items which are irrelevant for their classification this is the number of items on е rec- 
into predominant patterns. dividual patterns K and L agree. It is their ie ind 
ment score, out of a total of 120 umes. < H-I 
genus level, we find the codes K-L and 
TABLE 2 
_ АМ ILWISTRATIVE CIASSIFICATIQI OF DESCRIPTIONS CF "BEST" MECHANICS 
Genet K-L-H-I 
Agreenent score: 86 
Species: K-L Rene 
eement score? 106 pecies: He 
Agr ES Agreement score: 10) 
"Ho" Anse: h, 8,32 33, e 
0, 61, 70, 75, 89, 10: "Ho" Anse? 5, 15, 52, 
19, 61, 70, 75, 89, 103112 EA AEN 
"Yes" Anse: E: 5, 26, 31, PA i 
و‎ 77, 80,111, 116 ев" Ans.t Д, 9, LO, 19, 
d $0, 59, 66, 81,96,12 
i Pattern: K Indivii i 
Individual Pa EM E : individual Pattern: 1, Individual Pattern: Н Individual Pattern: I 
"Yes" Anse? 9, Й Yes" Ans.t 50, 55, 59 " 6, 2» 
E ря BE, EA Ф 59, Yea" Ans.: & 2 эз, —— „бу г) E 
ET, 96, 100 & dmm & & 
"o" Anse? 59 "Ho" Anse: = 3 
Her dn ё ge 210 0 "Ho" Ans.: В, 26, зу mio" Anset 2, 2D pA 
zd at Bee = & D 
E E 89,103 10, 12 
ak 
Capital letter = codes for patterns. 
Double underlined пшлЬегз = agreement scores, 
her numbers = codes for items. The code mmbers listed imm sent, 
ТӨР tend for which both individual phtternscf и селе оча below each of the two species, for example, m YE 
pattern of the other species has the opposite answer, In cases where pF Ts but for which at least опе TP ios 
have the opposite answer, the code mmbers are underli; еге both indivi, 


ned, 
dropped at the next higher classification as irrelevant, 


items, These itens are listed under both individual paite, 
the individual patterns of the species agree on the other 


The two indivi 
rns K and L, 
106 itens of 


dual patterns of the other SPP 
listed et each level which аге |) 
Cual pattern of species K-L disagree оп... 
but with opposite answers in the two plac 
the inventory, 


Only those itens are 


pr. 


We 


A PA 


letters mean that this genus includes 
Species. patterns K-L and H-I, which were de- 
rived from individual patterns K, L, H. and I. 
Ihey agree on 86 items as shown by the twice 
underlined number, 86, listed under the code 
letters for the genus. Other code letters and 
other twice underlined numbers of Table 2 are 
interpreted in an analogous fashion. 

The species are arranged from left to right 
according to the size of agreement scores with 
the largest ones on the left. In other words, as 
We move from left to right across charts such 
às the one of Table 2, the classifications generally 
become less dependable, simply because we thus 
generally obtain successively smaller agreement 
Scores, and they are successively less dependable. 

In a complete chart of results (which could 
be prepared from the analysis), all of the items 
and responses to them would be listed in such 
a manner as to show both the exact answers con- 
tined in any pattern, and which items differ- 
entiate various individual patterns, species, 
Benera, etc, one from another and to what 
extent, These points can be illustrated by refer- 
ence to the incomplete chart of Table 2. Con- 
sider the items listed immediately below cither 
of two species which classify in a genus, The 
items below either of these two spccies are the 
ones for which cach individual pattern of the 
Species has the same answer, and for which at 
least one individual pattern of the other species 
has the opposite answer; sometimes both in- 
dividual patterns of the other species have the 
Opposite answer. In this latter case the code 
numbers for the items concerned are underlined. 
The items are classified according to whether the 
agreement is in terms of a "yes" answer or a 
по” answer. For example, both individual pat- 
terns of species K-L have a "у on the list of 
items beginning with 2, 26, ete. and both 
individual patterns of species H-I have a “no” 
оп the list of items beginning with 5, 15. 5% Cle 
Items 5, Во and 116 are common to these two 
groups, as indicated by the single underlines. For 
these two species, the latter three items are per- 
fectly positively correlated with each other and 
Perfectly negatively correlated with the other 
underlined items of the species, ic items for 
Which species K-L have “no” answers and species 
H-I have “yes” answers. 

A complete chart would show the entire de- 
Scription contained in each individual pattern, 
Such Species pattern, each genus pattern, etc. 
This can be illustrated by reference to the in- 
complete chart of Table 2. Suppose that we want 
10 know the composition of individual pattern L. 
> he answers to the items listed immediately 
и it give onlv a few of the answers which it 

cludes; some of the others are the answers to 
the items listed for the species containing the 
Still others are 


letter L, namely species K-L. 
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the answers of the items for genus K-L-H-1. 
These latter answers are not shown in the incom- 
plete chart of Table 2 because of lack of space. 
\nalogousiy, as would be shown on a complete 
chart, all of the additional answers of individual 
pattern. L. would be listed for the family, order, 
clas. and phyla which contain the letter L. 
‘The classification was completed at the phylum 
level because all classes came together іп a ingle 
phylum. The description of any other initial 
pattern or any species, genus, family, etc., сап 
be read in an analogous fashion from a complete 
chart. We see then that there is a very unusual 
and important feature of a complete chart, Even 
though it is the terminal table of a method 
which has analyzed data in order to facilitate 
interpretation of it, no piece of information is 
lost; the response of each subject to each item 


is shown. 

The above described feature of retaining 
every piece of information in the summary table 
(derived from the analysis) is not realized with- 
out creating a problem, One purpose of scientific 
inquiry is to abstract that material which leads 
to generalizations. If all information is retained, 
no abstracting has occurred. However, abstract- 
ing can occur at rious levels. In agreement 
analysis, cach successive level of classification 
represents an increase in the degree of abstrac- 
tion. Conscquently, the question arises as to 
which level yields the most meaningful results. 
From a purely practical point of view, we can 
argue in favor of that level which yields the 
closest relationship with an external criterion. 
However, this principle would appear to be a 
rather tenuous one on which to base funda- 
mental, scientific decisions. A better scientific 
basis сап be suggested if the problem is con- 
sidered in the light of the theory out of which 
agreement analysis developed, This theory postu 
lated real types. If real types exist, they may be 
either concise and discrete ог more amorphous 
in nature. If they are discrete, then they should 
reveal themselves most cogently at some one level 
of classification. This would probably be the 
level of classification at which the agreement 
scores have the highest statistical significance. 
Consequently, statistical theory of significance 
could be applied in an effort to determine the 
level of classification in which real types are por- 
trayed. Some statistics designed to evaluate the 
significance of agreement scores have been re- 
ported elsewhere (4). 

On the other hand, suppose that types exist 
only in a relatively amorphous sense; those at 
any опе level are ill-defined and fade into those 
at the next higher level. Under these conditions, 
there might be no very fundamental basis for 
stating that the types at any one level are more 
basic than those at any other level. The level of 
primary concern would then have to depend on 
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some utilitarian consideration, such as the degree 
of relation to an external criterion. 

There is another unusual feature of артее- 
ment analysis. Since the analysis is based on com- 
binations of answers, and since no combination 
whatsoever is excluded by the method, the 
method automatically handles whatever kinds of 
interrelationships (linear, curvilinear, and dis- 
junctive) are involved in the data, without in 
any way restricting them. The method also allows 
any specific answer to be given various meanings, 
or interpretations, depending on the combina- 
tions or patterns of other answers with which 
it occurs, 


Reliability of the method. In order to 
investigate the reliability of the method 
of classifying subjects into categories, the 
100 subjects of the experimental sample 
who were originally classified on the 
basis of the first 120 items of the in- 
ventory were again classified on the basis 
of the last 80 items of the inventory. In 
the original analysis, the 54 descriptions 
of "best" mechanics and the 46 descrip- 
tions of "poorest" ones were classified 
separately, This same approach was fol- 
lowed here in classifying the subjects on 
the basis of 80 items, thus resulting in 
two independent studies of reliability, 
one on descriptions of "best" mechanics 
and the other on descriptions of “ 


poor- 
est" mechanics. 

Each analysis was continued until every in- 
dividual was classified into one of two large cate- 
gories at the top level of classification, However, 
we did not wish to assess reliability at this highest 
level exclusively, because as already pointed out 
we planned tentatively to study later the validity 
at all levels of classification and ме wished to 
know whether or not the classification was suffi- 
ciently dependable to justify validity studies, For 
this latter purpose, it did not appear necessary 
to study reliability at all levels, if we could gain 
an overview from selected portions. Consequently, 
for the purpose of comparing the classifications 
on the basis of 80 items with the earlier ones on 
the basis of 120 items, we selected categories 
which were intermediate in the analysis. The 
categories were selected so that every subject was 
classified once and only once for the 80-item 
study, and likewise for the 120-item study, In 
the case of “best” mechanics, we selected for 
both studies the first class (two orders) of 32 sub- 


jects resulting from the analyses and the two 
remaining orders of 16 and 6 subjects. In the 
case of “poorest” mechanics, we selected the first 
order (two families) of 16 subjects and the four 
remaining families of 8, 8, 6, and 8 subjects, The 
categories of “best” mechanics were ordered in 
each analysis according to the number of test 
items on which the patterns of cach category 
agreed. In the 120-item analysis of "best" me- 
chanics, the 32-subject class agreed on 43 items, 
the 16-subject order on 31 items and the G-subject 
order on six items. These categories were desig- 
nated 1, 2, and з respectively. In the 80-item 
analysis, the 32-subject class agreed on 36 items, 
the 16-subject order on 24 items and the 6-subject 
order on 13 items. These categories were desig- 
nated a, b, and c respectively. 

To the extent that categories a, b and c have 
common members with categories 1, 2 and » re- 
spectively, the method of analysis is reliable 
across items. The correspondence between these 
two sets of categories is shown in Table 3. Thirty- 


TABLE 3 
Association BETWEEN Two AGREE 


ANALYSES OF COMMON SUBJECTS 
("Brsr" MECHANICS) 


Categories | Categories From First 120 Items 
From Last 
8o Items 3 2 2 
а 1 8 23 
b x - 8 
с 4 т I 


four of the 54 cases, 63 per cent, have identical 
classification in the two analyses. Combining 
Categories 3 with 2 and b with c to obtain ; 
2X3 table (with larger expected frequencies 
уе computed the chi square and found it to à 
5-12, significant at about 0.09. (Chi squares " 
5-41 and 3,84 are required for p—002 2! 
b = 0.05 respectively.) ies of 
The analogous results for the 5 categories 
the “poorest” mechanics are shown in Table 4: 


TABLE 4 


Association BETWEEN Two AGREEMENT 
ANALYSES OF CoMMON SuBJECTS 
("Poonrsr" MECHANICS) 


Categories 


Categories From First 120 Item? 
From Last - 
8o Items 8 1 6 5 4 
4 8 
e ï 4 3 
Ў 5 5 2 4 
А i Р 
h 5 E à 5 1 


—— 


А 


| 
| 


A^ 


| 
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Twenty-four of the 46 cases, 52 per cent, have 
identical classifications in the two analyses of 
“poorest” mechanics. Combining categories 8, 7, 
and with 4: f, g, and h; and d with e to 
obtain x2 table (with larger expected fre- 
quencies) we computed the chi square and found 
It to be 16.81, significant ас better than 0.001. 
(The chi square for p = 0.001 is 10.83.) These 
two sets of results show that binary agreement 
analysis gives reasonably dependable results even 
when applied to relatively homogencous groups 
such as “best” or “poorest” mechanics. 
Validity of the method. The two cla 
fication schemata from the original analy- 
sis based on 120 items (one for patterns 
descriptive of 54 "best" and the other 
for patterns descriptive of 46 “poorest” 
mechanics) were investigated as tools for 
differentiating between cross-validational 
ls " 
samples. In order to simplily a discussion 
of the method used, suppose we have an 
individual pattern. X, which is not one 
of the experimental patterns, and sup- 
a iras ” 
Pose its categorization as between “best 
and “poorest” is kept a secret until we 
classify it in terms of the two schemata. 
We then check to see if the classification 
is correct. In order to classify pattern X 
11 terms of the two schemata, we first 
Obtain an agreement score for it with 
each of the individual patterns of the 
€xperimental samples. On the basis of 
these agreement scores, we classify indi- 
vidual pattern X in one of three ways, 
namely, (a) as a pattern descriptive of a 
46 ig : еен 
best" mechanic, if its highest agreement 
Score is with an experimental "best" in- 
dividual pattern, (b) as a pattern de- 
Scriptive of a "poorest" mechanic, if its 
highest score is with an experimental 
“ H - М p 
poorest" individual pattern, or (c) as 


indeterminate, if there is a tie and indi- 
vidual pattern X has its highest agree- 
ment score with both a "best" and a 
"poorest" experimental, individual pat- 
tern. The classification. schemata аге 
valid for individual pattern X if they 
classify it correctly. 

However, in the above procedure, we 
have not yet exhausted all of the po- 
tentialities of the classification schemata. 
In fact, we have hardly used the schemata 
at all. We have scored individual pattern 
X in terms of individual patterns only. 
We can also score it in terms of the pat- 
terns of species, genera, families, etc., for 
"best" and "poorest" experimental data 
and classify it as either "best" or “poor- 
est" at each of these levels. A problem 
is to discover which level gives results 
in closest agreement with the criterion 
classification. "This problem is investi- 
gated by classifying many cross-valida- 
tional, individual patterns as "best" or 
"poorest" at each level and analyzing the 
results to determine which level gives 
the highest percentage of correct classifi- 
cations. We did this for the individual 
patterns of our cross-validational sam- 
ples. 

The results are shown in Table 5, 
which reports the percentages of correct 
classifications for cross-validational, in- 
dividual patterns descriptive of “best” 
and “poorest” mechanics, shown sepa- 
rately, апа then also combined, with 
each category weighted equally. For ex- 
ample, at the species level, 100 per cent 
of the individual patterns for “best” 


TABLE 5 


PERCENTAGES OF 


Correct CLASSIFICATION ON THE Cross-VALIDATION SAMPLE (120 ITEMS) 


Genera 


Families Classes Phyla 


ИД! v Orders 
ation Individuals Species ders 
« ! Е тоо тоо тоо тоо xos5 
st" [ 100 
ist sin | ee 83 76 70 70 33 1 
oorest хо > 88 85 85 67 57 


93 02 


Combined 
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mechanics and 83 per cent of those for 
“poorest” mechanics were correctly classi- 
fied. A mean of these two, namely 92, 
is the average percentage of the combined 
groups correctly classified. The maximum 
percentage of correct classifications for 
the individual patterns descriptive of 
"poorest" mechanics is 89 and occurs at 
the individual level. For those descriptive 
of “best” mechanics, it is 100, for all 
levels above that of the individual pat- 
terns. For "best" and "poorest" com- 
bined, and equally weighted as groups, 
the maximum percentage of correct 
classifications is 93 and occurs at the 
individual pattern level. 

Multiple types of “best” and "poorest" 
descriptions. There is one very interesting 
characteristic of the cross-validational 
percentages. The largest percentage for 
the individual patterns descriptive of 
“poorest” mechanics is at the individual- 
pattern level, with 89 per cent correctly 
classified; the percen tages then show a de- 
crease down to only 33 per cent correctly 
classified at the class level. In other words, 
as descriptions of both "best" апа * 
est" mechanics are classified separately 
into more inclusive categories, the items 
in terms of which the "best" and “poor- 
est" descriptions differ drop out of the 
analysis. These items which differentiate 
as between “best” and "poorest" descrip- 
tions also differentiate within both the 
"best" and “poorest” groups and conse- 
quently yield many types of both "best" 
and “poorest” descriptions. On the 
other hand, the items in terms of which 
many "best" descriptions agree have 
identical or similar responses for descrip- 
tions of “poorest” mechanics; "poorest" 
descriptions do not differ from “best” 
ones primarily in those characteristics 
which are common to "best" descriptions, 
but rather in those which are relatively 


poor- 


heterogenous for even "best" descrip- 
tions. Our analysis revealed по single 
ideal type of “best” mechanic which dif- 
fers from an all-inclusive prototype of 
"poorest" mechanic; multi-types seem to 
be required if maximum differentiation 
is to be obtained between categories of 
"best" and "poorest" mechanics. 
Hypotheses analogous to the ones just outlined 
have derived from a similar study by the ed 
which compared mental hospital patients ide 
community subjects (5). Mental hospital p 
do not seem to differ from community saneca 
on those characteristics which are common : 
almost all community subjects. Instead they UA 
pear to differ more on those characteristics W nit 
are peculiar to species and genera of common y 
subjects. Multi-types of both patients and 9 Don 
munity subjects appear to be required in cae А 
to achieve maximum differentiation. ps ch 
potheses might prove helpful їп areas Дит 
have frustrated quantitative efforts to i 
entiate categories of individuals satisfactorily. 
; ion. 
Improvement through item select! З 
г correc 
The fact that the percentage of corr s 
gage rol 
classifications decreases as we move ft = 
۴ Р its s 
species to higher level categories ie iin 
the hypothesis that items which are is 
ies 
vant to the greatest number of spec 
- r- 
are the most effective in obtaining ve 
" ۴ items 
rect classifications, However, some € : 
А igher- 
are relevant for both species and hig e 
level categories. In fact, in order to = 
E ga ry, а 
relevant for à higher-level category» ев 
А i JES 
item must be relevant for all u 
: Аф x О re rele- 
classified in it. But items which аге ! 5 
А в His ove 5 
vant at both species and higher 1€ и 
x E N rd 
would fail to yield the species. In 0 sit 
* h А 1 
to differentiate between species, we m at 
have some items which are relevant. ng 
; ir 
the species level only. In attempt i 
item 
to select а reduced number of ate "i 
Which would be equally as ciet d + 
the original 120 items, we selectec am 
Items with the following characterist at 
x а 
(а) each item selected must be ane 
ес 
for at least 82 per cent of the SP ie 
isolated; (b) some of the items selec 


must be relevant primarily at the species 
level only; (c) others must be relevant at 
higher levels as well. As a consequence, 
the items selected varied in relevancy at 
the order level from those which were 
relevant for only 16 per cent of the 
orders up to those which were relevant 
for 71 per cent of the orders. 

Using these 26 items, in lieu of the 
Original 120 items, we repeated both 
the experimental and cross-validational 
Studies. As a control group of items 
against which to compare the above se- 
lected sample, we chose 26 items at 
random from the 120 original ones and 
also repeated the studies on them, 
_The percentages of correct classifica- 
tions derived from 26 especially selected 
items on the cross-validational samples 
is shown in Table 6. It is analogous to 


TABLE 6 
PERCENTAGES oF Correct CLASSIFICATIONS 
FOR CROSS-VALIDATION SAMPLES (26 
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TABLE 7 


PERCENTAGES OF CORRECT CLASSIFICATION, 
For Cross-VALIDATION SAMPLES, 
26 RANDOM ITEMS 


Ітемѕ or KNOWN RELEVANCIES) 
Classifi- | Indi 
ASS ndi- Spe- Gen- Fam- " 
cation | vidual ‘cles era ез Orders Classes 
PR" g 100 тоо тоо тоо тоо 
ез 8 8 8 72 20 
Combined] 09 E ES ie 83 $ 


Table 5, which was based upon 120 
items. For combined samples (“best” and 
, Poorest”) of the cross-validational sub- 
Jects, the 26 items give higher percent- 
ages of correct classification than the 
120 items at the individual, species, 
8enus, and family levels; at the order and 
class levels, the 120 items give the higher 
Percentages, The highest percentage of 
Correct classifications, namely 95, is ob- 
tained at the species level for the 26 
Items, 

The percentage of correct classification 
derived from the 26 randomly selected 
“tems for the cross-validational subjects 
are shown in Table 7. This table is anal- 


Classifi- | Indi- Spe- Gen- Fami- 

cation | vidual cies era lies Orders Classes 
"Best" 1co 100 100 100 тоо тоо 
“Poorest” 75 76 78 76 72 48 
Combined] So — $8 ŝo — 58 % 74 


ogous to Table 5, based upon the 120- 
item study, and to Table 6, based upon 
the 26 especially selected items. 

The 26 especially selected items gave 
the two highest percentages of correct 
classification, 94 and 95, and tied with 
the 120 items for the third highest, 93. 
In addition to this tie, the 120-item test 
gave the fourth highest percentage, 92. 
The 26 random items gave the fifth high- 
est, go, and tied with the 120-item test 
on the sixth highest, 88. 

Since the purpose of selecting items was to 
obtain those most effective at the species level, 
we computed the significance of the difference in 
percentage of correct classifications at this level 


between the 26 especially selected items and the 
»6 items chosen at random. Table 8 shows the 


TABLE 8 


Accuracy OF CLASSIFYING SUBJECTS WITH 
RANDOM VS. SELECTED ITEMS 


26 Items of Known Relevancies 
26 Random 


Items Incorrect Correct 
Correct ks 58 
Incorrect 5 7 


data for this computation, assuming a correlation 
between the percentages. However, this table 
does not meet the minimal requirements for a 
computation of this kind; one cell has a zero 
entry and the cell diagonally below it has only 
seven as an entry (2, p. 80). A test which fails to 
assume a correlation can, however, be applied to 
this data provided the result is interpreted with 
the understanding that it is an underestimate of 
the significance of the difference to the extent 
that the percentages are correlated, as they would 
normally be in a situation of this kind. A test 
of this kind (2, p. 76) was applied. It produced 


14 
a critical ratio of 1.88, which is significant at the 
0.03 level in a single-tail interpretation, as is 
here appropriate. 

The empirical results reported above 
support the hypothesis that items which 
have high relevancies for species and vary 
in their relevancies for higher-level types 
give higher percentages of correct classi 
fications than an equal number of ran- 
domly selected items. 


In cases where an agreement analysis fails to 
give a satisfactory percentage of correct classifica- 
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tions, there is sometimes a possibility of improv- 
ing on the results. possibility derives from 
the fact that one application of agreement analy- 
sis to all of the items of a test isolates only the 
predominant types. The responses which do до. 
fit these types are rejected at the various leve e 
of classification, The pattern. analysis could si 
repeated on the rejected responses. Iht se w 
yield species not isolated from the analysis of SiC 
test as a whole. ‘They might give correct classi 
cation missed by the other items. In the present 
study, we had relatively few incorrect oe 
tions and relatively few responses rejecied at fee 
species level, As a consequence, we did uot ji 
a reclassification on the basis of rejected. item»: 


TABLE 9 


ы Р » “ "= 39) 
43 ITEMS DESCRIPTIVE or THE First Crass (2 ORDERS) or “Best” MECHANICS (.V=3 


Item No. Response 


Ttem 


b " " Р | time. 
You don't have to worry about telling him what to do all the tin 


ys of doing things. 


г with him pretty well. her. 
an find out for you one way or another. 


job done. 


a job, you can always be sure he will get the 


things which are explained to him. 


Wrong than right in his decisions. 


ake a decision one way or the other. he docs. 
h experience know a lot more than Пе 
round until you're done before offering to help: 


rk, he doesn't do a thing. 
ttle step to him. 


and sometimes you can't. 


1 Y He can get along with people pretty well. 
11 Ү When he does a job, you know it will be done right. 
13 Y He answers questions as best he can, 
17 Y He knows how to use tools. 
18 Y His personal traits are О.К. 
39 Y You can take his word for what he says he's done, 
44 x He deserves a promotion. 
47 
57 Y He sees decision through to an end. 
бо Y He uses the right tools when he can get them. 
67 £ He is very glad to help somebody else. 
68 ¥ He makes sure he does a good job. 
72 bí He tries to find better w: 
73 M His work is nearly 100% correct. 
78 Y ‘The pilots all seem to get along 
85 Y What he doesn't know he с: 
до ¥ Everyone enjoys working with him. 
or Y He knows his stuff. 
97 Y If you leave him to do 
98 ¥ What he knows, he never forgets, 
тот ү 4 He usually remembers 
105 ¥ He can show you how to do the job right. 
106 Y His ambition will pay off. 
107 X He is good at working on the plane. 
тта Ў He gives good cooperation, 
119 Y Не seems to take pride in his work. 
7 N He is more often 
10 N He works in a sloppy way. 
2t N He is afraid to make a de 
2 N Most guys with that muc 
28 N He just stands а 
3o N He doesn't have any sense of responsibility. 
35 N He's just careless, я 
38 N He isn't a very careful worker, 
41 N He achieves his aim in the wrong way, 
46 N He 5 always getting messed up in money deals, 
бо N He is kind of slipshod in his w: Б 
74 N M hen he does get over to wor 
03 N You have to explain every li 
95 N No cooperation at all. 
113 N Sometimes you can trust him 
115 N Instead of doing it, he says, 
120 N 


“TI think a it 
Пе “goofs-off" all day long. tients sous it 
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а reclassification on the basis of the present data 
would not represent a critical evaluation of the 
Proposal, and it is too laborious to apply it to 
noncrucial data just because the latter are avail- 
able? 

In the results of the several analyses 
reported herein, it is helpful to mention 
that all of them agree in differentiating 
more “poorest” descriptions from “best” 
ones at the lower levels of classification, 
where there are relatively many cate- 
Zories and each one classifies only a few 
individual patterns, rather than at the 
higher levels, where there are few cate- 
gories and each classifies a large number 
of individual patterns under it. 


IL A Jom CONSIDERATION OF AGREE- 
MENT AND FACTOR-ANALYTIC RESULTS 


In this section types are interpreted in 
terms of results from both agreement and 
factor analyses. Then, the interassocia- 
lions between factors and types are 
studied further in terms of items which 
ате common and distinct for the types 
and factors. 


Interpretation of the Types 


Several of the primary advantages of 
agreement analysis are realized when we 
attempt to interpret the types. Each type 
is broad in the kinds of information fur- 
nished about the subjects who constitute 
the type, provided the test is wide in its 
Coverage. This point is illustrated by list- 
ing the characteristics which were found 
In this study to belong to the first class 
(32 subjects) of "best" mechanics iso- 
lated. These characteristics are shown in 
Table 9, with the items which were 
answered “Yes” listed first, and those 
Se 

"In addition, we have recently developed a 
method of multiple classification by agreement 


analysis whereby a subject can be classified in 
terms of all of the category patterns which are 


Contained in his individual pattern. 


answered “No” following. 

The diversity of characteristics repre- 
sented by these descriptions is illustrated 
by considering several of the items at the 
beginning of the above list. The first one 
tells about human relationships, the next 
one about quality of job performance, 
the next about reactions to questions, 
then about use of tools, personal traits, 
truthfulness, etc., in the order named, 
with each one giving information in a 
different area of behavior. 

The diversity of information furnished 
about each type is further substantiated 
by a brief comparison of results from the 
agreement analyses with those from a 
square root, unrotated factor analysis of 
common data. 


The 120 items which were analyzed here by 
two separate agreement analyses, using first 54 
"best" mechanics and then 46 "poorest" ones, 
previously had been factor analyzed using 428 
subjects (3), including those used in the agree- 
ment analyses. Using the results of this factor 
analysis, we examined the 43 items listed above 
as descriptive of the first class of "best" me- 
chanics. These 43 items represent substantial 
loadings on 13 of the 23 factors reported in the 
factor analysis. Each of these 13 factors contains 
at least two items from the above 43 with loadings 
in its upper ten, and nine of the 13 factors con- 
tain four or more items (from the 43) with load- 
ings in the top ten per factor, The first factor has 
all of its 10 highest loaded items represented in 
the 43 descriptions of the first class of "best" 
mechanics. The next two factors have none 
represented; the fourth has eight in the top 
ten, The others are as follows, with the fac- 
tor number listed first, followed by the num- 
ber of items from the first class with loadings in 
the top ten: 5-4, 6-5, 7-0, 8-5, 9-6, 10-0, 11-1, 
12-5, 19-1, 14-7, 1571, 16-5, 17-2, 18-2, 19-2, 20-0, 
21-1, 22-1, 23-2. This comparison shows that the 
first class covers a wide breadth of information 
in terms of the number of factors involved. 

Similar data for the last order of “best” me- 
chanics (6 subjects) described in terms of 20 
items, shows the following numbers of items 


4 Five of the factors had fewer than ten items 
listed because a lower limit for loadings of .100 
was applied. These five factors had 9, 9, 6, 7, and 
4 items respectively. Our comparison was re- 
stricted to the items listed. 
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TABLE 10 


" MECHANICS (V=6) 


Item No. Response 


Item 


UA AUAM eZ 


He can get along with people pretty well. 
He has a nice appearance. 

He knows how to use tools. 

When he works, he really works. 

He makes sure does a good job. 

He's just a “regular Joc.” " " 
If үч leave Ёш to etl job, you can always be sure he will get the job done. 
He's just a normal guy. 

He is good at working on the planc. 

He scems to take pride in his work. 

He doesn't have any sense of responsibility 

He thinks he has had a raw deal in the service. 

He's just careless. 
He achieves his aim in the wrong way. " 
You have to stand over him and show him how to do everything. 
He shines in making decisions. 

He occasionally drinks as he works. 

No cooperation at all. 

He isn't a credit to the Air Force. 

He “goofs-off” all day long. 


with relatively high loading on the factors, with 19-2, 20-0, 
the factors listed first, followed by the number 
of items: 1-9, 2-0, 3-1, 4-7, 571, 6-1, 7-1, 8-5, g-2 
10-0, 11-0, 12-2, 13-1, 14- 


28 ITEMS DESCRIPTIVE OF THE THIRD FAMILY or “Рр 


5 o a, Ў 
2, 15-1, 16-1, 17-1, 18-2, lows: 1-4, 2-4, 3-8, 4-0, 5-0, 6-6, 7-2, 8-0, 075» 


PTE w 
21-0, 22 23-1. Analogous T w 
P e aie Me. got chanics (8 sub* 
the third family of "poorest" mechanics e fol- 
2, jects described in terms of 28 items), 15 25 


« 
7-2 


TABLE i: 


OOREST" MECHANICS (№= %) 


Item No. Response 


Item 


10 
12 
23 
27 


RAZA A LL A, АУ a Pa Ba A Cig Py I a АНАНАСА 


He works in a sloppy 

You wouldn't feel 
He is no “whiz” in his line. ^ 
Most guys with that much experience know a lot more than he does. 
He isn't a very careful worker. 

When he does a job, you know it will be done right. 

He is one of those ideal mechanics. ; 
Many times he works overtime even when he doesn't have to do 80: 
He knows what any modification of a Tech Order is about. 

He is a good man on anv job. 

He deserves a promotion. 
You don't have to worry 
He continues on without taking many breaks. 

He will straighten a guy out and explain things to him. 

He seems just about the best. I've seen as far as getting his work 

He will do an overtime job instead of leaving it to one of the ot hers. 
He always has better ideas than most men s 
He makes sure he does а good job. ` 
He's the energetic type who has to have 
His work is nearly 100% correct, 

He knows all the systems on an airplane. 

If he were an instructor, he could put it over pretty thoroughly- 
He knows his stuff. 

If you leave him to do a job, you c 
He reads new Tech Orders at hi 
He keeps working as long as th 
He can show vou how to 
His ambition will pav off. 


way. 
е unless you checked behind him. 


+ . sti е. 
about telling him what to do all the tim 


done- 


something to do. 


" ‚ор done: 
an always be sure he will get the jol 


8 first opportunity. 


ere's a man left standing. 
do the job right. 
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18-2, 19-0, 
sons between 
ypes riptions of 
types cover a wide variety of information in 
terms of the factors involved. An additional ad- 
Vantage is that information is given directly 
about specified subjects. It is not necessary first 
to analyze factors and then assess cach individual 
in terms of the factors. 


The gains realized in terms of an 
agreement analysis are not obtained with- 
out a sacrifice. The types cover such a 
breadth of information that it is dificult 
to do justice to the description of a type 
other than by listing the items which are 
descriptive of each type. We have already 
listed the items of a class (including the 
first two orders) to illustrate one type of 
“best” mechanics. Another type of “best” 
mechanic is illustrated by the last order 
of "best" mechanics isolated. The items 
of this order are listed in Table 10, with 


those answered "Yes" listed first and 
those answered “No” following. 

Normally an order would have 16 
individuals in a binary agreement analy- 
sis. The above order has only six because 
48 of the 54 subjects in the sample were 
classified into the first three orders. 

As an illustration of a type of “poor- 
est" mechanics, the items descriptive of 
the third family (8 subjects) of poorest 
mechanics isolated are given in Table 
11. 

The items descriptive of the one addi- 
tional order of “best” mechanics 
(N = 16), the one order of “poorest” me- 
chanics (N = 16), and the three remain- 
ing families of “poorest” mechanics 
(N = 8, 6, and 8, respectively) are listed 
in Tables 12-16. 


The descriptions of the fourth family (Table 


TABLE 12 


р ORDER or “BE 


' MECHANICS (V=16) 


31 ITEMS Descriptive or THE THIR 


vou know it will be done right. 


ord for what he says he's donc. 


Hing him what to do all the time. 


ls when he can get them. 


vs of doing things. 


always be sure he will get the job done. 
are explained to him. 


з н Item 
Item Ҳо, Response 
А E X He wants to get in and do the job. 

ET ¥ When he does a job, 0 1 

13 x He answers questions as hest he can. 

Xa Y He knows how to use tools. z 

31 Y He always reports to work on time. 

39 Pd You can always take his w 

Б He deserves a promotion. , 

" Y You don't have to worry about te i 

57 Y He sees decisions through to an enc. 

ae s He uses the right tool 5 

be Y He makes sure he docs à good job. 

i 1 otter W 
2 E He tries to find bet уз of doin 

5 Y He stands up for what he believes in. 

97 Y If you leave him to doa job, youen à 
то! Y He usually remembers things W nen fe 
TOS Ж He can show you how to do the job right. 
106 ve His ambition will pay off. бе 
19 ч He is good at working on the plane. 
109 ME He likes being a mechanic. 

118 X. He gives good cooperation, , — ^ 
119 Y He seems to take pride in his work. 

| Ыбан 
is He works in a sloppy way. 


fraid to make 


Sometimes you 
He “‘goofs-oll 


to the Air Force. 
1 can trust him and 
' all day long. 


a decision one way or another. , 
{ until you're done before offering to help. 


and show him how to do everything. 
hat happens to him when he gets out of service. 


1 sometimes you can’t. 


N H 
М E 
28 К He just stands arounc 
35 N He's just careless 
53 N You have to st уе n 
75 N He doesn't seem to care w 
95 N No cooperation at all. 
I04 N He isn't a credit 
N 
N 
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14) appear to be, in general, complimentary of 
the mechanics even though the mechanics were 
selected as "poorest" ones. This apparent incon- 
sistency seems reasonable when we consider that 
the mechanics with whom we are concerned have 
been trained and selected for their jobs, and the 
terms "best" and "poorest" are relative and ap- 
plied in this setting to small groups of mechanics. 
Each supervisor, who served as an informant in 
describing a mechanic, was requested to select 
either a "best" or a "poorest" mechanic and de- 
scribe him. Each informant made his selection 
from those mechanics whom he had supervised 
in the last two years. Since the informants gen- 
erally supervised not more than six or seven men 
in any one assignment, the group from which 
each informant made his selections was small. 
And cach group, as already pointed out, con- 
tained only men who had been trained and se- 
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lected as mechanics. It is not surprising then to 
find that eight of the 46 men chosen as “poorest” 
mechanics are described rather complimentarily. 
They probably are reasonably good mechanics. 
This conclusion from the experimental sample 
is consistent. with the results from our cross- 
validational studies (Tables 6, and 8) where 
at least 11 per cent of the poorest" mechanics 
invariably classified as "best" ones in terms of 
our pattern analytic method. К 

However, there is a way in which the compli- 
mentary description of this family of "poorest? 
mechanics seems to differ from complimentary 
descriptions of "best" mechanics, The compli- 
mentary descriptions of “poorest” mechanics tend 
to avoid the most highly mechanic-centered de- 
scriptions, Instead, the compliments tend to һе 
limited to the more general kinds of descriptions, 
such as: "He is а good man on any job"; “He 


TABLE 13 


38 ITEMS DESCRIPTIVE OF THE First ORDER (2 FAMILIES) or “Poorest” MECHANICS (№=16) 


Item No. Response Item 
9 ря Не accepts responsibility only when he must. 
то Ү He works in a sloppy way. 
22 Y His superiors should not give him a higher AFSC. 
23 ¥ He is no “whiz” in his line. 
2 Y Most guys with that much experience know a lot more than he does. 

28 Y He just stands around until you're done before offering to help. 

30 Y He doesn't have any sense of responsibility. 

38 Y He isn't a very careful worker. 

Şi ¥ He does things without first finding out what will happen. 

74 p4 When he does get over to work, he doesn't do a thing 
115 Y Instead of doing it, he says, “I'll think about it." : 

II N When he does a job, you know it will be done right 

16 N He is one of these ideal mechanics. : 

17 N He pioa hor to use tools. 

2 7 € knows what any modification of a T i t 

A N реа BE ee 1 of a Tech Order is about. 

44 P ne on a promotion. 

47 Î ой don't have to worry about telling him what а time. 

48 N He continues on without taking see fon маи 

49 N He will straighten a guy out and explain things to him. 

50 N He seems just about the best I’ve seen as far as getting his work done. 

57 М He sees his decisions through to an end i | 

А N Everybody likes him. ` 

65 ? € will do an overtime job instea aving it ers. 

66 N He always has better diss ined ни ышанын 

68 М Не makes sure he does a good job. g 

72 N He tries to find better ways of doing things 

73 N His work is nearly 100% correct. ga 

79 N pe knows all the systems on an airplane. 

83 N T ше times when he keeps working even though everyone else takes 

5 N Everyone enjoys workin wi i 

N Не knows his stuff. EN: 

Е N you leave him to doa job, you can alway T job done- 
ipe N He reads new Tech Orders Д [f tele аца е эшан 
186 М Не can show you how to do the job right “> 
107 N He is good at working on the plane. | 
тїй N He gives good cooperation. 

119 N He seems to take pride in his work. 
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TABLE 14 
22 ITEMS DESCRIPTIVE or THE FOURTH FAMILY OF “POOREST” MECHANICS (N—8) 
| Item No. Response Item 
x If he were a crew chief, he would work right along with his men. 
ү He doesn’t just sit back with his hands on his hips. 
x He is a good man on any job. 
Y You can take his word for what he says he's done. 
¥ You don’t have to worry about telling him what to do all the time. 
Y He seems just about the best I've seen as far as getting his work done. 
Y He uses the right tools when he can get them. 
b He tries to find better ways of doing things. 
¥ He stands up for what he believes in. 
Y What he doesn't know he can find out for you one way or another. 
a ү Everyone enjoys working with him. 

Ү If you leave him to doa job, you can always be sure he will get the job done. 
P4 He's just a normal guy. 
N He's just careless. 
N He hàs unpleasant characteristics. 
N We have had a couple of “run-ins.” 
N He doesn't seem to care what happens to him when he gets out of service. 
N He is just a kid. 

93 N You have to explain every little step to him. 

ToS N He doesn’t care to mix with other people. 

112 М He is pretty unhappy about being in maintenance, 

115 N Instead of doing it, he says, “I'll think about it.” 


stands up for what he believes in"; "He is just the first class of "best" mechanics are 
а n өр "oct" ics o = E " "et 
к, отта! guy." The "best" mechanics on the included in the 20 items descriptive of 
her hand are often described in terms of items us в . 
the last order of “best” mechanics. In 


which pertain more specifically to their particu- , 
lar jobs as aircraft and engine mechanics, such other words, we have evidence here of 


as: "He knows how to use tools"; "He is good iwa quite distinct types of “hes? me: 


at working on the plane”; “He scems to take pride b 
chanics. 


in his work"; “He likes being a mechanic." | А 2 
| Only 10 of the 22 items descriptive of 


Only six of the 45 items descriptive of the fourth family of “poorest” mechanics 


TABLE 15 


18 ITEMS DESCRIPTIVE OF THE FIFTH FAMILY or “Poorest"’ MECHANICS (N=8) 


Item No. Response Item 


Y He doesn't know his job. . : F 
2 Y His superiors should not give him a higher AFSC. 

is no “whiz” in his line. : 

27 Y Most guys with that much experience know a lot more than he does. 
ү He always reports to work on time. P 

st Y He does things without first finding out what will happen. 

8o Y He has just an average working knowledge of the job. 


\ 84 Y He stands up for what he believes in. 
Y It takes him quite a while to learn. 
He is one of these ideal mechanics. 
He is a good man on any job. 

Че гез a promotion. : ч 
He SIT тей a guy out and explain things to him. | 

st about the best I’ve seen as far as getting his work done. 
better ideas than most men. 


e always has n 
n v an airplane. 


He knows all the systems on 
He knows his stuff. 


He reads new Tech 


N 
N 
N n 
3o N He seems just 
1 
n Orders at his first opportunity. 
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TABLE 16 


7 [ITEMS DESCRIPTIVE OF THE SIXTH FAMILY oF “Poorest” MECHANICS (N =8) 


Item No. Response Item 
9 x He accepts responsibility only when he must. 
23 Ў He is no “whiz” in his line. | р 
25 М He knows what any modification of a Tech Order is about. 
34 N He is a good man on any job. 
44 N He deserves a promotion. 
66 N He always has better ideas than most men. 
11$ N Instead of doing it, he s, "I'll think about it.” 


are included in the 43 items descriptive 
of the first class of "best" mechanics, and 
only two of the 22 items for the poorest 
family are included in the 90 items 
descriptive of the last order of “best” me- 
chanics; the numbers of items which are 
not common suggests that "best" and 
“poorest” mechanics differ in ways other 
than merely being opposite. 


Interassociations Between Factors and 


Ty pes 


In the above section on interpreting 
the types, comparisons were made with 
results from an unrotated, square-foot 
factor analysis of common data. In the 
present section, we wish to review the 
comparison for another purpose. We wish 
to call attention to interassociations be- 
tween types and factors. The present 
evidence as to the exact nature of the 


associations is not maintained to be con- 
clusive, and the data available are not 
the most crucial kind. The responses are 
descriptions by supervisors of mechanics, 
rather than known characteristics of me- 
chanics. Consequently, whether the in- 
terrelationships derived from them reflect 
only the opinions of informants or are, in 
fact, characteristic of mechanics, is not 
known with certainty at this time. The 
factor analvsis was unrotated, and by a 
square-root method rather than by a 
more exacting procedure. Likewise, the 
agreement analysis was by the binary 


method rather than the more exacting 
and complete procedure (6, 7)- 


Of the three types of "best" mechanics Te 
ported, the large class of them (32 subjects and 
43 items) is heavily involved in the first factor 1n 
that all 10 of the items with the highest loadings 
on this factor are contained in the 43 items 
descriptive of this type, Each of the next two 
factors (in order of amount of variance accounted 
for) have nonc of their ten highest-loaded items 
in the 43 items descriptive of the large class O 
32 subjects, This class had a disproportionate 
amount of its variance accounted for in the first 
factor, and consequently other types were !n- 
volved in the second and third factors. In the 
fourth factor, the large class reappears together 
with the other two types of “best” mechanic 
Eight of the ten highest-loaded items on this ш, 
tor are included in the 43 items descriptive °° 
the large class, and five of these plus two other 
of the ten highest-loaded items are among te 
20 descriptive of the last order of “best” де 
s. The relative association of these HYP 
with other factors сап be seen in the figures "e 
ported earlier which show the number of high- 
loaded items for cach factor from cach pom 
Similar results were reported for the third family 
of “poorest” mechanics (8 subjects and 28 пету, 
They reveal that this type is heavily involved a 
factors 3 and 21, with & high-loaded items е 
cach, The two "best" types reported had tle 
one item with a high loading in either of ШЕ. 
two factors, The high degree of association a 
tween factor 21 and this "poorest" type is © эг 
more striking when we consider that this ae 
accounts for only 0.8 per cent of the variance З 
the data ana ‹ valysis P 


46 of те 
was 


Уз ‘d, and the agreement а! 
poorest" mechanics was done on only b 
428 subjects on which the factor analysis 
performed, 


factors WE 


In this comparison of types with 3 
s of 8° 


have illustrated our points with one clas “pest 
subjects and one order of six subjects from E 
mechanics, making a total of 88 “best” mechan 
and leaving one order of 16 “best” mechan? 
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una d 7 i 
naccounted for, Analogously, we have used only loaded items common to a factor and a type are 
taken to mean that a type is relatively high or 


б шш, RE "poorest" mechanics, leaving four 
pues, Se cial mechanics and one of six me- low cn а factor. For example, the four items 
nted for, Data analogous to that — listed under Factor and opposite Type 1 mean 
this "best" type is relatively high on this 


ш reported are summarized for all of the the 
above- i d "mes Yr Т" rud 
e-mentioned types in Table 17. This table fac or. Analogously, the four items under Factor 


uc coda 23 factors which were isolated inthe 8 and opposite Type 4 mean that this “poorest” 
Gr а square Toot solution, listed in order type is relatively low on this factor, The stand- 
then eed iance accounted for. All of ings are all in the direction which would be 
actors except the roth, 14th, goth, gend, expected from the names of the factors and the 
ич 23rd had ı0 items with loadings above 0.100. labels of "best" and “poorest” which are applied 
т specified factors bad only 0,9. 6, 7. and 4 to the types, except for Type 6, the fourth 
A Хө ызды, with loadings above 0.100. family of "poorest mechanics, which is com- 
ane 1% concerned with the 10 highest-loaded plimentarily described as we have already 
fun к, пш ary Le ns e poned ош. á ; | 
E a abo oft лоо. F > + ү i ү 
it is concerned only with p се pe The en tees seas Be ee tae ini 
ible. alis Ims toot: redet ади "podré сане wd beni. uà P "et ey are 
elem ae ypes of "best" and "poorest те not lazy or weak in character. They do not fail 
Thera s Чати ed from the agreement analysis. to use knowledge effectively, are industrious, re- 
liabilit um the particular one used in the re- member well, are socially acceptable, are practi- 
ilis ЫШ study; reported earlier. For each ype, cal workmen and do not lack craftsmanship. Type 
B8 кешеа Shows the number of high-loaded items 4 subjects, on the other hand, are low on re- 
Hed factor. The table re als, as already sponsibility and willingness to work; they are 
ustrated, that types are associated with factors. lazy and fail to use knowledge effectively; they 
auc 17 contains an additional lesson. It аге not industrious and do not remember well; 
indi s Way in which factor and agreement they are not practical workmen or craftsmen, 
favi Ping be used jointly in interpreting be- and they lack both intellectual capacity and 
- For this purpose, four or more highly- job knowledge. This "poorest" type is defined 


TABLE 17 
or Елси ТҮРЕ ARE DISTRIBUTED AMONG THE 


How Іт :SCRIPTIVE 
FACTORS IN TERMS OF THE Hicu LoADINGS* 
= 
Factor Titles and Numbers 
> 
| 2 Z 
Hi g E! 
5 E E 
5 3 a 
z 2 P > 
E E a A B S = 
Bos 2 HE 3s 8g 37 
йат „ > 222288 $ 
S 2 924 858 а пш 82 з 5 
2554 88 5 Е ER Ég S 5 
El 6 28654 2g дая 32172959 $ 
Sle £ S 2. Ru sea 5 £ = ل‎ ешо ® 
ele 229345 § 328 823 58 L5 3 t 
aie 2 SOUS 3 гПа СЕБЕ ке шч 
8 süSgseipiiisfBdioiii:ui3 
als ¢ 88 елед 8s 5 22 2 8 3 À x MER 
АГ НЕЛЕ БЕ ЕКЕ КЕК КЕ КЕЕ 
" eala 22232 325528 SEERR R44 855 
5 2-2 АД E i i t 
E ЧОО КЕ ede ee eae 
E gigis vue ee ad guess 
1 “Best” 
—1з/ Class 8 2 © ват жашта B 2 2 2 от т 2 
— 4 2 o S 4 5 3 2 
Sard Order S 32 тө 9 иа 4 9 b os: оз X o s 9 $t GEE = S 
34th Order | 5 | о ae * ug л ond TEL à X B® 1 1 i $ @ © ¥ o I 
“Poorest” -— 
4—15 Order £ Box: g 4 1 T T o 4 71 5 2 3 1 о 6б o 1 
{зи Family 38 1 3 b $32925352292922 6 4 12008 1 о 
4th Family | 2 8| 44 3 à» 9i. rto» m eto £ 4d à X 1 RS 1 
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are the same items which have the 


s of “best” mechanics, то of them 
rms of loadings on factor 2; the same is true for factor 3; 


д, etc. All of the factors except the roth, 14th, 20th, 22nd, and 23rd had 1o items 
ad only о, 9, б. 7 and 4 items respectively with loadings above о.тоо. In the case et 
ad only o, 9, & апа just specified. In all other factors, the ten items with the то 


ribe the rst clas 


a 
43 is among the first ro in (е 


high oT example, of the 43 items which dese 
eight ofan loadings on factor 1; none of the 4 
With loade, 43 аге among the first то for factor 
these 0291065 above о.тоо, These latter factors h 
hi actors, the comparisons were limited to the num! 


i 3 
shest loadings were used. 
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as opposite to the above “best” type in all 
but two factors (weakness of character and social 
acceptability) of the eight used to define the 
“best” one. In addition, the “poorest” type is 
defined as deficient in three factors which were 
not used in defining the “best” type (willingness 
to work, intellectual capacity, job-knowledge). 

In an analogous fashion, the other types can 
be described concisely from Table 17. In de- 
scribing these types, it should be remembered 
that the raw data are descriptions given by 
Supervisors; the patterns of descriptions may 
exist only in the minds of the supervisors or 
both in their minds and in the behavior of 
mechanics. Consequently, the patterns are not 
known at this time to be characteristic of the 
behavior of mechanics. They are here reported 
only as characteristic of descriptions of mechanics 
by supervisors, Additional research is required to 
determine whether they would also be obtained 
from objective assessment of behavior, 

We have shown that some items are internally 
consistent in both factor and pattern analytic 
analyses, However, there were nine items which 
were internally consistent for some of the types 
by agreement analysis but did not appear with 
substantial factor loadings on any of the factors. 
These items are as follows: 


6 If he were a crew chief, he would work 
right along with his men. 


18 He answers questions as best he сап, 

15 He will climb into one of the airplanes 
and sleep. 

34 He is a good man on any job. 

BB If there is something he likes to do, 
he does it faster than anyone else would. 

57 He sees decisions through to an end. 

67 He is very glad to help someone else. 

73 His work is nearly 100% correct. 

75 He doesn't seem to care what happens 


to him when he gets out of service. 


The one thing that these items appear to 
have in common is that they are extreme state- 
ments It seems that some items can be highly 
dependable for describing some one or more 
categories of subjects even though they are not 
generally dependable for describing all subjects 
in terms of any one factor. 

We searched for items which had relatively 
high loadings but were not included in any of 
the types on which the reliability study was done. 
(The search was made at an intermediate level 
of classification.) Relatively high loadings were 
defined as sufficiently high so that an item was 
included in interpretation of a factor (either in 
the first ten items in terms of loadings for a 
factor or a loading above .100 in the case of those 
factors which did not have 10 items with loadings 
above this value). There were go items with 
relatively high loadings (as just defined) which 


did not appear in any one of the types on 
which the rcliability study was done. In addition, 
there were 28 items with relatively high loadings 
which were not included in descriptions of 
"best" types and 25 other items with rclatively 
high loadings which did not appear in descrip- 
tions of "poorest" types. These three categorics 
of items plus the nine items already reported 
(which showed maximum relevancies for the 
types even though they had no relatively high 
loadings) make a total of 92 items on which there 
was considerable difference in thc assessments 
of internal consistency by the two methods 
(agreement analysis and factor analysis). t 

Even though we grant that relative size 9 
factor loadings on unrotated square-root йг) 
treated singly, is not a good estimate of nami 
consistency, the large discrepancies here рене 
show that there is a difference in the interna 
consistency of items as measured across all sub- 
jects (factor analysis) versus its assessment а 
selected categories of subjects (agreemen 
analyis). In addition, these results show that 
many items differ in their validities for азса 
factors and types. In short, these findings SP 
size a new way in which reliability and val: zd 
are relative concepts; they are relative to ы 
assessment across all subjects versus their assess 
ment within selected categovies of subjects. " 

The above results reveal the песа for a s 
crucial comparison of results from factor "ihe 
agreement analysis of common data, using o 
most refined technique of each method, toget > 
with data especially selected. for the purpose 
the comparison, 


ПІ. A СагпоцЕ or тик METHOD 


Tt will be helpful now to react critically oe 
method just outlined and applied. The ee 
is made computationally feasible by an pried 
restriction; there is no reason (other ane 
simplicity of classification) why the types а! two 
one level should be composed of only re 
Patterns from the next lower level. voids 
Complicated version of agreement analysis S rer 
this arbitrary restriction (7). However, this 18 n 
method is laborious and it is felt that thc Peeing 
binary method is probably sufficiently еха© 
for our present state of knowledge. 

Another problem of the method is th 
по level of statistical significance which an аЁ 
ment score must meet before a classification ? es 
be based on the score. The method circums ors 
this problem by the following logic: It ae 
types, and argues that if they exist ү je 
patterns of response, then every pattern 0. 
Sponses must be classified and a solution Ж, a 
give each pattern its best classification. This / 
much less complicated solution than would ec 
involved if the level of significance were requin 


at it sets 
тсс" 
ап 
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lo be computed. for every classification of every 
Pattern, 

One could wonder whether or not “yes-no” 
answers are sufficiently dependable as a basis for 
Classification, A reasonable reply to this kind of 
Inquiry is to state that we do not have sullicient 
Information of an appropriate kind on which to 
Studies on the reliability of 
answers have been based on their de- 
Pendability as assessed across all subjects of a 
ample, Agreement analysis recognizes the possi- 
bility of differen reliability of an answer; 
an answer mav be highly reliable and dependa- 
ble for some categories of subjects and not others. 
Agreement analysis enables us to determine 
Whether or not “yes-no” answers have sufficient 
reliability of this kind to continue to justify the 
{pplication of the method to them. Even if 
Yes-no"™ 


base an answer, 
Ves-no 


answers were found wanting, the 
Method might still prove helpful if applied to 
Categorized d; 15 obtained by dividing scores 
on several ables into class 
Intervals, 


continuous vär 


Agreement analysis does not allow us to assi 
“very type а point in space in relation to every 
ee type, and thus to give it a number which 
Would reveal its location, Whether or not one 
Wishes to do this depends on one’s theory of the 
Nature of types, If one wishes to assume that 
the same response in two different patterns has 
the same meaning, then it is reasonable to use 
the overlapping responses of patterns to а ign 
"pes locations in space: this can be done in a 
Method called linkage analysis (8). However, if 
One Wishes to assume that the meaning of a re- 
SPonse depends to а considerable extent on the 
Pattern in which it occurs, then there would 
1рреаг to be no basis for allocating the types to 
зр al locations on the basis of overlapping re- 
Sponses in their patterns, 

е Agreement analysis does not settle the differ- 
ees dn theoretical issues about types. и їз, 
к а method which should prove Ы = 
ing gs along with other methods in P igat: 
in si me of the theorie about types, am е 
"dying Whether or not it seems worth while 
9 assume 


types. 
IV. SUMMARY 

This paper has applied an abbreviated 
Version of a method of analysis, called 
"Breemen, analysis, for the isolation and 
ч ification of types into a hierarchical 
System, An unusual feature of the method 
15 that the terminal table on which in- 
‘rpretations are based contains every 


› ан : 
It of information represented in the 
Tàw q and 


ata: information is abstracted 


vo 


generalized at successive levels, and the 
level most pertinent for a given purpose 
can be selected for further study. In 
addition, the method of analysis can 
handle all kinds of interrelations in data 
(linear, nonlinear, and disjunctive), and 
it also provides for giving various inter- 
pretations to any particular answer to an 
item, depending on the combination or 
pattern of other answers with which it 
occurs. 

Illustrative applications of agreement 
analysis to descriptions of mechanics 
selected as “best” or “poorest” confirm 
the following hypotheses: The method 
yields both reasonably dependable and 
valid results, Descriptions of "best" me- 
chanics can be differentiated [rom those 
of "poorest" ones on the basis of type 
membership. In addition, the results sug- 
gest the hypothesis that categories of 
people, such as "best" and "poorest" me- 
chanics, can be most clearly differentiated 
in terms of many types, each type con- 
taining only a few highly similar sub- 
jects, rather than a few types containing 
many subjects. In terms of our data, 
there appear to be many types of "best" 
mechanics and many types of "poorest" 
mechanics—but no all-inclusive single 
' mechanic which differs 


type of "bes 
from an all-inclusive type of "poorest" 
mechanic, Hypotheses of this kind can 
be further investigated by the method of 
agreement analysis which has been out- 
lined here. 

Results from our agreement analyses 
were compared with results [rom a factor 
analvsis of common data. The compari- 
son "illustrated that the two methods 
classify data differently but not entirely 
independently; the factors and types 
which resulted are interassociated. Both 
methods yield results which appear to 
be meaningful and they can be used 
jointly in the interpretation of behavior. 
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However, some test items can have in- 
ternally consistent responses for some 
categories of subjects in terms of an 
agreement analysis even though they are 
not sufficiently consistent across all sub- 
jects to yield any factor loadings of any 
consequence. These items appear to have 
value for selecting out rare subjects, such 
as a mechanic who is so expert that he is 
always 100 per cent correct; or, on the 
other hand, a poor mechanic who is so 
worthless that he would climb into an 
airplane and sleep. 

Conversely, other items can be so in- 


ternally consistent across all subjects that 
they have relatively high loadings and 
still not be retained as descriptive of 
any one of the predominant types at 
intermediate levels of classification. Con- 
sequently, the internal consistency 0! 
many items in terms of predominant 
types is quite different from their im- 
ternal consistency in terms of factors. 
These issues on reliability of items under 
the two methods of analysis require fur- 
ther study by the most refined techniques 
of factor and agreement analyses with 
items especially chosen for the purpose. 
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taken to achieve a product (i.e., to solve 
a problem), or the correctness of the 
product, or similar static indices have 
traditionally been the criteria by which 
performance was evaluated. А severe 
limitation on our level of understanding 
of such behavior has been imposed by 
the use of such methodologies; we are 
denied access thereby to information 
about the way in which factors within 
the performance itself contribute to the 
outcome, and the way in which these 
factors are related to other variables than 
problem-solving performance. 


It is possible to achieve solution to a 
problem efficiently or inefficiently or even 
fortuitously. It is possible to fail bril- 
liantly. In order to understand the dy- 
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namics of performance, to analyze the 
factors which relate to the achievement 
of a solution in a particular way, to in- 
vestigate the possibility of relationships 
between variables external to the prob- 
lem-solving performance and certain as- 
pects of the process, it is desirable to 
make accessible for direct observation 
the problem-solving process itself. This 
requires the development of a technique 
which will permit the externalization of 
the dynamics of the process, such as the 
relationship between the product and the 
state of information of the subject, in- 
stead of continued reliance primarily on 
techniques which observe product and 
rely on introspection or subjective re- 
port to relate that product to its ante- 
cedent causes. 


Recognition of this is implicit in some of the 
theoretical work in the problem-solving area by 
Duncker (2) and others who have attempted to 
analyze problem-solving as a process in order to 
understand the relationships between successful 
performance and factors within the behavior it- 
self. Theoretical and conceptual schemes have 
been developed about typical behavior in various 
situations, and facilitation or hindrance of suc- 

ss. These attempts have been constrained by 
reliance on recorded verbalizations or qualitative 
descriptions by an observer of the behavior, In 
recent years, there have been a number of 
promising attempts to develop an experimental 
methodology which would permit more oper 
tional definition of the elements of behavior 
and more precise quantification of these cele- 
ments. To the extent that such attempts are 
successful they will enable development of 
theoretical schemes which can be subjected to 
experimental verification, 

Recognition of the necessity and the desir- 
ability of analyzing process rather than product 
is explicit in the excellent monograph on prob- 
lem-solving by Bloom (1), The reader is recom- 
mended to read this work, if he is not already 
familiar with it, since it contains not only 
cogent arguments for preoccupation with devis- 
ing process measures but a large number of ex- 
perimental findings which are confirmed їп 
more quantitative terms by the present work. 

Some attention has been given to various ver- 
sions of the game of Twenty Questions, by Har- 
low (5) and a number of other workers, as a 
source of process information. These approaches 


493... 


have arisen from concern with military training 
devices or trouble-shooting analyses, and have 
tended to deal with procedures for the repair of 
faulty equipment of one sort or another, where 
checking procedures are assumed to be binary ор- 
erations which eliminate cach time one half of 
the remaining alternatives, While such inquiries 
are of interest, their utility tends to be limited by 
the structure of problems which permit such 
binary operations. Of perhaps more general in- 
terest are techniques developed for very similar 
purposes but without this restriction in struc 
ture, such as the Multiple- Alternative Symbolic 
Trouble Shooting Test (MASTS) of Grings et al- 
(4). which is yielding some information about 
previously inaccessible of the trouble- 
shooting process, in electronic problems. Ag Hs 
the highly specialized nature of the task limits 
the utility of results so far obtained for under- 
standing of more generalized problem-solving 
proce: 
Of greater generality, and therefore реті f 
greater potential utility in the investigation Q: 
the problem-solving process, is the Tab ge 
technique, recently developed by Glaser ч = 
(3). While this task was developed for te 
analysis of electronic trouble-shooting profit iana 
like those above, it possesses a flexibility P 
permits its application to many other ай пе 
inquiry, The technique consists of mer 
a subject with a problem and a set of Jue in 
items of information potentially of utility" 
the solution of the problem, These items ich 
selected. by the subject. in any order paar 
scems pertinent to him, until he has galli 
those items which he requires for the ш 
of the problem, The sequence of selection lex 
these items by the subject is used as the ene 
of the way in which already acquired inform ia 
tion is used, The authors of the Fab. Item "е 
cate their recognition of the flexibility of 1 
technique in as of more general CON say 
than electronic trouble shooting when: ШИ тт 
“Proficiency in medical and psychological a que 
nosis might be tested by adapting this (ш E 
to measure the ability of a clinician 10 Ре! ар: 
the procedures necessary for correct к 
nosis... . In general, the Tab Item 15 а be- 
plicable technique for the measurement O 0 
havior which involves the serial performante 
а set of procedures where the performance 
one procedure vields information. which 
Plies a cue for the selection of the pes 
Subsequent. procedures.” Rephrased, 018, no 
nique might be of utility in the investiga os c 
problem-solving where the information n Jem 
from examination of one aspect of the pro? 


aspects 


ps of 


tech- 


. IY 
can „һе used to direct subsequent inqui those 
Specific other aspects of the problem unti solu- 


f the 


aspects nece ry for a stipulation О 
tion to the Problem are understood. 
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cate Paepae Rimoldi (7) has de- 
a ап applied a technique very similar 

зе Tab Item for the investigation of the 
s of solution of problems of medical diag- 
this dd анн analysis. As in the Tab Item, 
(5 3 SEIN consists of presenting a problem 
m кш апа then making available a body 
BP ieee uen. containing items which are more 
Тенг а to the solution of the problem. 
tine es are labeled so that information con- 
diste pru, aspects of the problem сап be 
tice Чү ıe subject chooses, The order and 
recone 1e items selected. by the subject are 

ded, 
Я е Өр referred to above represent 
ology ee orward in problem-solving method- 
Writer Ei a faeces appear to the present 
used to Pas T from a number of drawbacks. As 
ing ШЕ: к ley ате restricted 10 problem ‹ 
knowledge їйї» a high degree of specialized 
шшш үз к the consequence that the process 
testes tq E dependent on the cducation of the 
exceedin a ie ee extent. In addition, it is 
content oe stants to stipulate the information 
OE do eri is items available to the subject. 
items, E 5 * ще information content of these 
item с уаде the extent to which each 
Problem а иез to ап understanding of the 
the vane a whole. Perhaps most important, 
Thar na Gf puse problems is not realistic. 
lo an a Ж structure the path of the subject 
Шиши desirable extent by supplying a limited 
Lee i eee alternatives, give away in- 
and ania ni а the possible alternativi 
Performed. cues and checks which can be 
sionality IE TE the additional dimen- 
actually ка 7 ? results from having the subject 
abstract $i, atik real events, rather than their 
where Nein representations, in a situation 
of Жеке ion is possible, and where the set 
or Жай actions is not artificially delimited 

in time. 


p BOUES utility, the ideal format 
Ro у of the problem-solving 
СЕДАН should start the subject with a 
a т minimum of information about 
бте TE and then require him to struc- 
а mj Ща own presolution behavior with 
A UM of externally imposed coit 
on ES Such a format should be Завой 
«бын О which is maximally Ln 9 
Perien Skills, special knowledge, or ех- 
Since ces peculiar to a given «Ше 
minis is important that repeated s 
i ation of comparable forms be pos- 


Sible be. 
' $0 that changes in process after 


various kinds of intervention can be 
measured, it is also desirable that the 
effect of familiarity with the generic task 
be minimal. It is vital to be able to 
quantify the difficulty of the task, and to 
be able to vary the difficulty over a large 
range, as well as to know the informa- 
tion contained in any element of the be- 
havior, Finally, it is desirable to devise 
a format which presents the subject with 
the necessity to interact with real events 
rather than abstract. 


A technique which permitted the direct ob- 
ation of the problem-solving process and met 
the above stipulations would offer a number of 
advantages. It would permit analysis of the prob- 
lem-solving process itself to see whether it was 
qualitatively homogeneous, or whether it is com- 
posed of qualitatively distinct phases which 
could be defined and described. "Theoretical for- 
mulations, such as those of Duncker, for in- 
stance, could be evaluated against such informa- 
and modified if needed. The factors deter- 
the relationship between such phases, 
ansition from phase to phase, 
a such observation, The re- 
experience, cul- 
ables, and the 


serv 


tion 
mining 
and affecting the tr: 
might emerge fron 
lationship between education, 
tural background, personality 
internal factors which interact in the prob- 
lem-solving process might be clarified. The 
extent to which particular aspects of the process 
are characteristic of an individual might be de- 
termined. The reorganization in process which 
might result as a consequence of change in 
the severity of the demands of the task on the 
individual could be analyzed. The extent to 
which various techniques or experiences are of 
value or of utility in increasing the effectiveness 
of an individual's problem-solving behavior 
might be evaluated, and knowledge gained about 
how to change specific aspects of the behavior. 
Relationships between such aspects of behavior 
and skills or abilities involved in real-life ac- 
tivities might become evident, enabling better di- 
rection of the acquisition of these skills. Diag- 
nostic examination of the problem-solving be- 
havior of an individual might permit application 
of remedial techniques to the areas of process 
constrain that individual's per- 


which most 


formance. 
The ultimate realization of these possibilities 


would require extensive research, once suitable 
instruments. for the observation of process were 
devised. The achievement of the ideal format 
described earlier represents а difficult task in it- 


self. 
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This paper describes а technique, 
briefly reported previously (6), which it 
is felt constitutes a step in the correct 
direction. While numerous criticisms can 
be directed against this technique in that 
it does not fully meet some of the desid- 
erata, yet it provides much information 
previously inaccessible and should be of 
use in the development of still more ap- 
propriate methods, and in the acquisi- 
tion of basic knowledge about problem- 
solving processes. 

In considering the results obtained to 
date using this technique, reported sub- 
sequently in this paper, we must empha- 
size that we are here dealing with prob- 
lem-solving performance of a particular 
kind. In these problems, a finite and 
enumerable set of discrete relationships 
must first be elucidated and then manip- 
ulated so as to achieve a single stipulated 
result, There clearly are problems which 
do not fall in the category represented 
by our technique; problems such as those 
involved in certain kinds of research in 
the sciences would appear to be rather 
similar to our generic task, while prob- 
lems such as those involved in the mak- 
ing of artistic products, where there is 
no right answer, or the elucidation of 
less discrete relationships using probabil- 
istic evaluations, would appear to be 
quite dissimilar, although perhaps anal- 
ogous methods can be devised for their 
investigation. One should not fall into 
the error of assuming that performance 
on this task represents evidence of some 
sort of basic ability which is involved 
in all sorts of productive activity. This 
word of caution is made necessary by the 
many examples of misapplication of 
"tests" of reasoning ability in selection 
procedures currently. 

Further, there is ап unquestionable 
element of stress and competitiveness in- 
herent in the mode of presentation of the 


task, as will be described in the follow- 
ing pages. Evidence that subjects are 
susceptible to this stress and consequently 
display a performance which by no means 
represents their optimal behavior has 
been gathered in a preliminary experr 
ment, and imposes a further constraint 
on conclusions about the relationship be 
tween "real" ability and the sample 
elicited in our task. 

While we will present evidence that 
behavior in the situation which we have 
constructed can be interpreted as an ap- 
proximation of the subject's habitual ар” 
proach to problems which fall in this 
category, and is meaningful as such, yet 
it is apparent that by appropriate instruc 
tion or experience over a period of time; 
or by diminution of the threatening aS 
pects of the situation, this habitual ар" 
proach can be modified.* The extent to 
which such modification would be ge" 
eralizable to analogous real-life tasks 15 
a problem of great practical importance: 

However, in our desire to qualify the 
interpretation of results gathered wi 
this method, let us not obscure our belie 
that we have here defined a real and m 
portant class of problem, which pe 
appear to be relevant to à number 


I 
?TThree individuals were tested in а а 
situation separately on one problem. They aged 
then brought together into a group а" с was 
to solve a second problem. The procedur enter 
for each individual to hand to the exprime ore 
the next step which he would make i ss 
working alone, and then for the group t° i was 
its next move until unanimous agrecmen cts 
achieved. The situation compelled the ne 
to formulate verbal arguments for their 
sals which would persuade the other ™ 
of the group. In this situation radically mbers 
behavior was exhibited by the group i 
from their previous performances, bre con- 
not compel the use of language with me, о 
Sequent imposition of syntactical structu arried 
their “reasoning.” This experiment WAS apn 
out with the collaboration of Mr. Richar the 
of the Mental Health Research Institute © 
University of Michigan, 
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real-life situations, and that this method 
gives access to information about prob- 
lem-solving behavior previously unavail- 
able. 


Description of the Problem-Solving and 
Information Apparatus (PSI) 


Problems can be conceived of as con- 
sisting of two aspects: elements and re- 
lationships. The logic of the relation- 
Ships between elements can be stated in 
terms of symbolic logic, devoid of factual 
content. Thus it is possible to construct 
an abstract logical problem with the 
Same [ormal relational structure аз à 
problem which involves factual content. 
Any logical relations, or propositions, 
Which can be stated in symbolic logic can 
be stated as an equivalent electrome- 
chanical circuit. It is possible, if one con- 
fines oneself to problems of a structural 
complexity commensurate with the num- 
ber of circuit relationships available, to 
Construct an apparatus on which a large 
number of sets of logical propositions 
Сап be stated. The difficulty (complex- 
ity) and structure of these sets of propo- 
Sitions can be quantified as desired, and 
€asily varied by the experimenter. A 
Siven set of propositions constitutes a 

Problem.” The achievement of a par- 
ticular output from a network of electro- 
mechanical circuits, which are related in 
accordance with such a set of logical 
Propositions, requires that the constraints 
Imposed by the logical relationships as 
defined by that set of propositions be 
Satisfied. The achievement of such an out- 
Put constitutes a solution to the prob- 
lem, 

, In accordance with the foregoin 
Siderations, an apparatus has been con- 
Sttucted (6) and a technique devised 
which permits observation of aspects of 
the problem-solving process which fall 
mto three categories: 


g con- 


1. Factors which relate to the amount of effort 
required and the characteristic habits of work 
displayed in the process of achieving the desired 
output from the apparatus. 

2. Factors which relate to the acquisition and 
handling of information in the process. 

3. Factors relating to the organization, 
manipulation, and synthesis of acquired infor- 
mation in order to achieve the desired output. 


This apparatus will be designated as 
the Problem-Solving and Information 
Apparatus (PSI) in what follows. 

Using the PSI, numerous aspects of an 
individual's problem-solving performance 
can be defined and measured quantita- 
tively in each of these three categories. 
The definitions of these variables will be 
presented in the next section. 

Information theory permits the pre- 
cise quantification of the total informa- 
tion content, ог "difficulty," of any set 
of propositions, or "problem." One can 
construct a number of problems of the 
same or graded difficulty. Such problems 
can be administered serially to ascertain 
the effect of increasing difficulty, or the 
extent to which learning occurs in this 
situation. The technique permits se- 
quential analysis, making possible a 
treatment of problem-solving as a branch- 
ing stochastic process with nonindepend- 
ent transition probabilities. In other 
words, one can determine the extent to 
which the sum of all the information 
obtained by a subject in the first n — 1 
steps of a given performance influences 
the nth step. The utility of this will be 
discussed further in the section on var- 
iables. 

The PSI itself consists of a set of elec- 


mechanical elements arranged, with 
ean com- 


tro 
other components, into a Bool 
puter. Desired logical relationships be- 
tween the elements can rapidly be estab- 
lished by means of a plugboard in the 
rear of the apparatus. A given set of re- 
lationships defines a problem. Problems 


are constructed so as to have a unique 


6 


soluuion. A solution is defined as the 

production of a particular output from 
the network of related elements by using 
three particular elements, called “input 
elements,” in some combination or tem- 
poral sequence. In other words, the ap- 
paratus can be conceived of as composed 
of three divisions: input elements, an 
output, and a network relating the input 
elements to the output. The task of the 
subject, then, is to learn how to achieve 
a stipulated output from the network of 
related elements, by using only the input 
elements. This requires that he analyze 
the logical relationships between the ele- 
ments and then utilize these relation- 
ships to produce the desired output. In 
all of the problems constructed to date, 
the required output and the set of input 
elements have been the same. Problems 
differ only in the structure of the net- 
work of relationships between the input 
and output, Front and rear views of the 
apparatus are presented in Fig, 1. 


On the face of the PSI is a display panel. On 
this panel is presented a circular array of nine 
pushbuttons paired with a circular array of nine 
lights. There are nine basic electromechanical 
elements in the network, and cach light indicates 
the state of one of the elements, The correspond- 
ing pushbutton permits the activation of that 
clement. Inside the circle of lights is a disc on 
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which is presented ап array of arrows connect- 
ing various elements. An arrow between Est 
elements indicates the existence of a relation- 
ship between those two elements, with the direc- 
tion of the relationship indicated by the head 
of the arrow. All arrows on the disc stand for 
relationships in the logical network, and all 
relationships which exist in the logical network 
are indicated by arrows on the disc. In the 
center of the disc is a white light, which is the 
desired output from the network. Clearly, ij 
cach particular problem, defined by а set 0 
rclationships, there corresponds а particular 
problem disc. | E 

Although the existence of all relationship iS 
indicated on the disc, the specific nature па 
relationship is not. An arrow from "A" to s 
for instance, might mean (a) that А was + i 
ficient to cause B, or (b) that A was nece ш 
but not sufficient to cause В, or (c) that A ч 
sufficient to prevent B. Sometimes the nature na 
a relationship can be uniquely inferred. simply 
by inspection of the disc, knowing that all rela- 
tionships indicated exist and that all relation- 
ships which exist are indicated, together with the 
given fact that all problems are soluble. td 
limes inspection alone docs not permit on 
drawing of a valid inference without ambiguity. 
In such a case, the subject must devise a p 
cedure to elicit the necessary information - 
the network, ‘The subject may аена, бе 
nature of any relationship he desires by же 
ing the pertinent elements by means of their 
associated. pushbuttons and observing the gre 
of consequences of this activation as displayed 
by the lighis on the panel, In other words the 
subject designs small logical experiments, from 
the interpretation of which he may infer the 
nature of the relationships among the elements. 
He may use all the elements as many times od 
in as many combinations as he desires, in orde? 


Fic, 1: a. Front of PSI apparatus, with exam 
plugboard and probl 


ple disc in place, b, Rear of PSI apparatus, showing 


€m ready to be inserted, 
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Example problem disc. 


Fic, 2: 


А the information which is necessary lo 
vedi the achievement of the required йиш 
У тЫ the permitted input айн, | - 
experi n of the order, nature, and time of these 
k ments is the raw data for the sequential 
зан ей to above and described in the 
tie EA an example may be useful to clarify 
the a which the subject faces and to We rur 
from mh an which information can be тт 
disc А; Tr Figure 2 shows the pro um 
mii Пав been used to date (0 КШ 
The E ay of the subject with the PS Д 
10 by Р: the subject is to learn how x 
Wing Mitis Pee or sequence 0 
h амь М hich involve using only 4 5 
Sequence shout learn what this combination 
Westions usi Should be, the subject may ask 
nor Dr ир, any of the g elements in any 
Ot ask па саол He may also, if he wishes, 
5» or idis Which use elements other than 
рат "lation, attempt. to achieve 10 simply by 
doge hie these three elements. In the 
Soli; this pa 1 was used in the work reported 
que d in miim: Subjects were asked to achieve 
Чо Stions of the п s time and using minimum 
Is inefficient s hine, in which case permuta- 
Way t us work S though permitted, behavior. 
are 11 Which th te example to illustrate one 
three arro на Subject might proceed. There 

WS which lead to light 10, the 


is c А i 
ties, tomar Crcially available, with or with- 
Ca A Som 5 recorder, from Standard Elec- 

Бо, lino; апу, 6728 S. Halsted Avenue, Chi- 


required “ouiput.” They come from 1, 2 and 3. 
‘These arrows might mean that z alone, 2 alone, 
3 alone, т © 2 but not 5, 1 © 3 but not 2, 
2 ёз but not т, огт е2 © 3 together is the 
necessary wad sufficient condition (NASC) for 
m Qo oam; yw KRALL WAH, Te MEEK 
AMARE DAN ORI TCD т 
one of the foregoing musi be We dn A 


ascertain which of these possibilities applies in 
this Case, we must ask questions of the PSI. 
Pressing the pushbutton т on the panel will 
cause the light which indicates the state of r 
to be lit on the display panel. If z were the 
NASC for то, after three seconds r would go 
out and light zo would be lit on the panel. 
If r is not the NASC for то, after three seconds 
light z will go out and no light will be lit on 
the panel. In our example, the NASC for light zo 
to be lit is z & 2 but not 5. This means that 
r & 2, if lit together, will cause zo to light if 
3 was not also lit. By the appropriate manipula- 
tions of the PSI, the NASC for zo are thus 
determined, From this point on, it is possible 
to solve the example without the necessity of 
asking any further questions. Let us see how 
this can be accomplished, and at the same time 
let us illustrate how the necessary information 
for the solution. сап be achieved by asking 
questions of the PSI, if one does not draw the 
inferences alrcady possible. 

Let us assume that we have asked the neces- 
sary questions to learn that r & 2 but not 3 is 
the NASC for 10. We now must learn how to 
light 1 & 2 without 3 by some combination or 
sequence which involves only 4, 5, and 6, to meet 
the criterion for a valid solution. Let us first 
ow to achieve the ability to light 1 in 
one arrow comes (0 1, and it 
1Ё 1 can be lit by any element, 
t clement must be 7. Since this example; like 
problems, is stated to have a valid 
1 cause 1. If we do not draw this 
sk what the arrow from 7 to 1 
n 7. Light 7 will be lit, 


consider h 
this way. Only 
comes fom 7. 


tha 
all permitted 
solution, 7 тиў 
inference, WC uen a ies 

y pushing 1 
barge coins later will go out as light A. 
lit Three seconds later light 1 will go 2 А " 

"will not be lit, because 1 is necessary DU 7 

к e for 10. The problem of how to light 1 
iare be restated as the problem of how to 
nel va ince We have learned that 7 is sufficient 
uy Two arrows come to 7, опе from 8 
n is > n 9. This might mean that 8 alone, 
ert: is the NASC for 7. 
о alone, 


x » о together, 
е ge SI which of these 
We can, 


k the P. i 
by pushing the various combinations and 

js true d 

observing the results. However, 


if we look one 
ү rrow comes to 
we sce that only one 2 
urther, we se т 
is : only one arrow comes [0 0. Both of thi «i 
д me from 4- If the arrow were to stan 
arrows OF revents, it would be mean- 


ationship $” ^ : 
e worm element is permitted to 


if we wish, as! 


for th 
ingless because n 
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cause 8 or g within the rules, since no other 
arrows are indicated. We have stipulated that all 
arrows on the disc must stand for real Telation- 
ships. Thus we can rule out the possibility that 
4 prevents 8 or 9. Similarly, 4 can not be neces- 
sary but not sufficient for either 8 or 9 because 
there is no other arrow to these elements. Thus, 
4 must cause both 8 and g. Now, if it is impossi- 
ble to achieve 8 without 9, and vice versa, we 
can rule out the possibility that either one can 
prevent the other from causing 7, because if that 
were the case this problem would have no solu- 
tion. Thus we conclude that 8 & 9 is the NASC 
for 7, which is the NASC for 1, and we can 
achieve 8 & 9 from 4. 

We have now learned how to achieve one part 
of the NASC in a fashion which is permitted in 
the requirement for a solution, which the reader 
will recall prohibits the use of any other cle- 
ments than 4, 5, and 6. We must now learn how 
to achieve 2 without 3, in a similar fashion, Two 
arrows come to 2, one from 5 and one from 6. 
This might mean that 6 alone, 5 alone, or 5 & 6 
together is the NASC for 2. We can ascertain 
which of these relationships actually holds by 
asking the necessary questions of the PSI, or we 
can take an additional factor into account. We 
know that 3 will prevent 1 & 2 from lighting 10, 
since the NASC for 10 is r & 2 but not 3. Only 
one arrow comes to 3, and it comes from 5. If 
this arrow meant prevents, the relationship 
would be meaningless, since no other element 
can cause 3. Similarly, it can not mean that 5 
and some other element are together necessary 
for 3 to occur, because there is no other arrow 
to 3. Thus, 5 must cause 3. If this is true, then 
if 5 & 6 together is the NASC for 2, 2 will always 

be accompanied by 5, and the problem cannot 
be solved. Since we have stipulated that the 
problem is soluble, 6 must be the NASC for 2. 
We can not infer validly whether or not 5 also 
is sufficient for 2, or whether 5 prevents 6 from 
causing 2, on the basis of the information we 
possess. This information is irrelevant, since we 
can solve the problem without the necessity of 
resolving this ambiguity. Of course, the ambigu- 
ity can be resolved simply by pushing 5 alone 
and observing the consequence, which is that 2 
is lit by 5 as well as by 6, 

We now know the meaning of every relation- 
ship in the network. We know how to achieve 1 
using only the elements permitted in a solution, 
and similarly we know how to achieve 2 without 
3. To achieve 1 we must use 4, which will cause 
8 and g, which will cause 7, which will cause 1, 
To achieve 2, we must use 6, not 5, because we 
must prevent 3 from accompanying 2. However, 
we must achieve 1 and 2 at the same time, since 
1 and 2 together is the-NASC for 10. A moment 
of thought will make clear to the reader that if 
4 and 6 are pushed together, after three seconds 
8 and g and 2 will be lit, then 2 will go out and 


8 and 9 will cause 7, and when 7 causes 1, 2 wall 
already have gone out. In other words, we have 
a temporal aspect to this problem. We must 
arrange our manipulation of 4, 5, and 6 into 
sequence which will give us the various parts О 
the NASC at the same lime. In order to do this, 
we must use 4 to cause 8 and Q to occur, war 
until 8 and g have caused 7 to occur, use б at 
that time to cause 2 at the same time that 7 
causes 1, and now we have established the NASG 
in a way which meets the restrictions imposed by 
the definition of a valid solution. 


The example contains all the kinds of 
relationships of which problems on the 
PSI are composed. In the example, i 18 
not necessary to ask further questions 
once the necessary and sufficient win 
tion (NASC) for 10 is known. Not al 
problems, of course, have this character" 
istic. An enormous number of different 
problems can be constructed from 
various meaningful combinations к 
these relationships which the design = 
the PSI permits. The information к 
tent of these problems can be rigorous y 
evaluated, and the content of cach P. 
sible question which a subject might 6 
of the PSI can also be evaluated, in " 
fashion which will be discussed in det? 
in the following pages. 

It should now be apparent th 
subject, as a result of his manipul 
of the pushbuttons in the course 
vestigating relationships, is рге 
with ordered sequences of events 1 5 
are consequences of his activation A 
elements. His task is to extract, from p 
servation of the sequences which he e 
signs, that information which he ne hë 
in order to be able to ascertain ie 
characteristics of a valid solution (0 е 
problem. Such a solution may acne 
an exceedingly complex ordering & 
combination of the input element: 


at the 
ations 
of in- 
ented 
yhich 


es 
ich enst 
Note: The three-second delay which are 


nt 
before the consequences of any antecede ject 


е b 
displayed has been imposed to give e m 
the opportunity to see the full set O їабс5, 


: init 
quences of any change of state which he in! 
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as an order 5 
afford b rl I of events, and as well to 
Manipulations which for him to execute any 
any point in ate nich he may desire to add at 
ет сес ЕЕ In preliminary studies, а 
complained abe ay was provided and subjects 
Second delay eee the excessive delay. A three- 
opportunity to ordi empirically to provide the 
causing the 2 order events adequately without 
Wait, and is ject to be annoyed by having to 
; ecause of oo a satisfactory compromise. 
ation of the m variation in rate of manipu- 
section on vari; I, described under Rate in the 
the ieee which follows, it is felt that 
various тше спа delay is not the cause of the 
t is, of pe henomena which have been found 
Providing a ind possible to check this point by 
y the Él as Gat m delay operated 

, but this has not yet been 


done, 
Some T 
models DN may be familiar with the 
extensively € nets which have been 
and Pe nlii by neurophysiologists 
to ToueeBtial e It may be of assistance 
Mechanical alize the PSI as an electro- 
direct A ж шы model, in which 
tion саң : tion, summation, and inhibi- 
ashion, 5. combined in any desired 
appreciate perhaps, the reader may 
3» which i; ает clarification from Fig. 
em on th ^ Bal equivalent of a prob- 
Senteq or S PSI, as it could be repre- 
the ing ace the subject has elicited all 
е Ormation necessary to stipulate 
к m ons Note that once the 
the кеја tained knowledge about 
; а oon he is still left with 
Nowledge al task of utilizing this 
Problem to produce a solution of the 
Familian, required by the rules. 
p hieveq ie with the PSI is 
Г atire pu the rules, describing 
lonships Ж es possible logical rela- 
aj Counter n | ше subject might expect 
ш these Ds and then illustrating 
zm detail y presenting the compre 
win onsec example which we have 
to, Paper ES The subject is provided 
М € notes. зш and encouraged 
thes effective} ung children are able to 
the кене ЫП the PSI, indicating 
on itself offers little diffi 


PROBLEM 1 


IF IT 15 TRUE THAT: ~ 


1 alone couses nothing 6 alone causes nothing 


2 alone causes nothing 8 olone causes nothing 


3 alone causes nothing 9 alone causes nothing 
4 olone causes 7 plus 8 
Salone causes 2 

7 alone causes 3 

3 plus 5 causes 1 plus 2 

6 plus В causes 9 

5 plus 9 causes nothing 

3 plus 5 plus 9 causes 1 

1 plus 2 causes nothing 

2 plus 6 causes nothing 

1 plus 6 causes 10 

1 plus 2 plus 6 causes nothing 


THEN, 


CAUSE 10, BY SOME COMBINATION OR SEQUENCE WHICH 


USES ONLY 4, 5, and 6 


Fic. 3: Verbal equivalent of first problem. 
Note: This represents the problem once all rela- 
tionships have been analyzed. 


ehension when illustrated 


culty of compr 
d of in the manner used 


concretely instea 
above. 


JI. PROCEDURE AND SCORING 


A. General Considerations 
nt of data is made avail- 
gle performance on this 


e many possible ways 
se data. 


A large amou 
able from а sin, 
here аг 
and examining the: 
ent from the subsequent 
it is not difficult to 
ich is a new way of 


apparatus. TI 
of organizing 
As will be арра! 
list of variables, 
define à variable wh 
looking at some aspect of the data. The 
possibilities [от such new variables have 
by no means already been exhausted. 
Particularly, the definition of variables 
which give access to the dynamic aspects 
our data without emphasizing only 
has been difficult. Our 
h has been in terms of 


of 
the static aspects 


habitual approac 


10 


products rather than process, and, pre- 
sented with precise and detailed data 
about process, one tends still to describe 
the product rather than the process. It is 
hoped that, as work in this direction 
»roceeds, a set of more adequate process 
indices will be developed. 

Many of the variables defined are 
reasonably independent, many covary 
appreciably with others. In view of the 
differences in performance of groups of 
diverse training and background, which 
will be illustrated in the data to be pre- 
sented, it would appear unprofitable to 
investigate the intercorrelations between 
these various variables at this time. The 
evidence suggests strongly that one 
should use a homogeneous population in 
order to determine intercorrelations be- 
tween the variables, in view of the prob- 
ability that the internal organization of 
the process, and therefore the correla- 
tions to be achieved, will vary as the 
structure of the group varies. Changes in 
situation and instruction may also 
change the internal organization. This 
pilot study did not attempt to obtain 
such information. Perhaps ultimately 
factor-analytic methods should be used 
to select a set of variables which span 
the space of the performance. However, 
the intuitive meanings and significance 
of many of these variables have been so 
useful conceptually that we have made 
no effort to prune our set of variables to 
a minimum. Such an attempt would 
appear premature until an appreciable 
period of examination of the many dif- 
ferent methods of organizing this some- 
what unfamiliar kind of data has 

elapsed. 

The variables so far defined and ex- 
amined can be grouped into three differ- 

ent major aspects of performance: 


1. Those variables which describe the indi- 
vidual's work habits and relate to the effort ex- 
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pended in the solution of the problem are re- 
ferred to as work variables. 

2. Those variables which describe the way in 
which the individual acquires and handles in- 
formation are referred to as information vari- 
ables. 

3. Those variables which describe thc orienta- 
tion of the individual to the problem, and which 
relate the effect of acquired information to this 
approach, are referred to as approach variables, 


B. Problems 


Subjects were presented with 
problems on the PSI. Both problems 
were administered to each subject sepa- 
rately in a single session, which usually 
lasted about опе and one-half hours, in- 
cluding time for familiarization and 
demonstration of the example. 

The two problems were constructed 
so as to be almost identical, with one 
major difference distinguishing them. 
Each contained 17 items of information 
in the total information pool; that is, 17 
simple propositions would define the 
network of relationships exhaustively. 
Each problem contains 4 relationships 
which are direct implication, 2 relation- 
ships which are conjunction, ı relation- 
ship which is compound conjunction 
and disjunction (x will occur if and only 
if y and z but not w has occurred), ап“ 
1 relationship which is disjunction. I he 
difference between the two problems - 
that the first problem requires that three 
coincidences be achieved in the course © 
the solution, while the second problem 
requires that four coincidences be 
achieved in the course of solution. One 
of the four coincidences used in the Lnd 
ond problem is the same relationship 
Which is required to produce a prior p 
incidence, but this relationship is US 
for a different purpose in the two i 
stances. Thus the difference between E 
two problems might be summarized pi 
stating that they are identical e 
that a relationship used only once in и 


two 
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first problem is used in two different 
Ways in the second problem, as a conse- 
quence of using diflerent elements in the 
same relations. 

Attention is directed to the apparent 
$mall difference between these two prob- 
lems, since presumably the differences in 
performance observed ате a consequence 
of these minimal differences in structure. 
Yet this minimal difference in structure, 
achieved without any increase in the 
number or nature of the items in the 
pool, brings about a major change in the 
behavior elicited from our subjects. The 
data presented below give us an insight 
into the extent to which small changes in 
Structure may change aspects of response 
which superficially one would expect to 
remain relatively unaffected. Clearly, the 
structure of relationships, as well as the 
sum of the information contained in 
these relationships, is a major determi- 
nant of the ease with which meaning, or 
Coherent organization of the set into a 
Whole which is unified, can be achieved. 
(One might suppose, since alternative 
Structures may lend themselves to the 
Same content, that clarification of this 
Point might generate understanding of 
the factors which constrain comprehen- 
Sion under various conditions of the 
Presentation of information.) 


C. Instructions 


In the experiments which yielded the 
ata to be here reported, subjects were 
asked to solve the problem in a mint- 
тит of time and with a minimum num- 
ber of inquiries of the PSI. The evi- 
dence strongly suggests that other instruc- 
Чоп, for instance to minimize time 
Or inquiries alone, or other situations, 
Which į shility that the 
subj ‘increased the probability Mt | 

ес А " ( 
Proces, Would use particula mediation 
SSes ¢ А 
to а es Such as language, might leat 
‘rent approach to the situation. 


alone 


This particular choice of instruction was 
made in an attempt to place the subject 
under some pressure to perform effec- 
tively while at the same time minimiz- 
ing the extent to which the situation was 
structured. It was hoped that this would 
enable us to observe the way in which 
our subjects themselves tended to struc- 
ture cognitive tasks of a highly abstract 
nature. 


It should be clear that the PSI permits the 
construction of other situations, and one can 
then compare the behavior elicited in these situ- 
ations to that obtained under the conditions 
cited above, Specific content can be put back into 
the problem, instead of using an abstract pres- 
entation. Problems can be presented with all 
relationships indicated and their nature speci- 
fied, Problems can be presented with none of the 
relationships indicated, Networks can be con- 
structed which are not constant, but which 
change as a function of certain aspects of an 
individual’s performance. Other instructions can 
be used. Betting behavior, or behavior under 
stress, or under various kinds of motivation can 
be observed. Group situations can be studied. 
Much can be learned about the dynamics of 
problem-solving behavior, and other kinds of 
behavior as well, by such variations. None of 
these interesting alternative approaches has been 
studied to date. In view of the many possible 
variations in the situation in which the PSI is 
administered and in the amount of content in- 
troduced into the particular problems selected, 
the information which has been gathered to date 
must be regarded as surveying only a small part 
of the domain which can be investigated using 
this basic technique. We will develop certain 
ideas and approaches to the analysis of this kind 
of sequentially observed behavior which may be 
of utility in the direction of subsequent studies. 
However, the results obtained with regard to 
differences between the performance of various 
kinds of groups and with regard to characteris- 
tics of the problem-solving process must be re- 
garded as gross outlines which must yet be filled 
in, rather than as definitive descriptions. This is 
ue in view of the small size of our 


particularly 
various groups. 
р. Subjects 

The subjects in these experiments were 
Chicago students and 
esponded to an ad- 
sons who 


q University of 


Al i 
di abers who! 
ME LAM 25028 
мес sinit offering to pay per 
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would participate in a psychological 
experiment. These students varied ap- 
preciably in their length of attendance 
at the University and came from various 
academic disciplines. On the basis of 
the criteria stated below, they were di- 
vided into groups as indicated. 

Group т: 21 Ph.D. candidates or recent Ph.D. 
recipients. (Composed of Group 3 and Group 4, 
below.) 

Group 2: 16 students in their first quarter of 
residence in the College of the University of 
Chicago. 

Group 3: 10 candidates for or recent recipients 
of the Ph.D. degree in the natural sciences. 

Group 4: 11 candidates for or recent recipients 
of the Ph.D. degree in other fields (8 social sci- 
ences, $ humanities). 

Group 5: 11 first-, second-, and third-year stu- 
dents in the College of the University of Chicago 
who indicated in an essay on career plans their 


intention of studying for advanced degrees in the 
natural sciences. 


Group 6: 11 first-, second-, and third-year stu- 
dents in the College of the University of Chicago 
who indicated in an essay on career plans their 
intention of studying for advanced degrees in 
areas other than the natural sciences. 

The mean performance of each of 
these groups will be presented for each 
PSI variable on both of the problems 
described in Section В. Our purpose in 
the remainder of this chapter is to pre- 
sent the definitions of the variables 
which constitute our present mode of 
analysis of PSI data, and to gain some 
insight into the extent to which the 
desiderata set forth on page 3 are 

achieved in the PSI. By discussing the 
relative performance of the six groups 
of subjects we hope to gain some idea of 
the dependence of PSI performance on 
the level of difficulty of the problem, on 
familiarity with PSI problems, on the 
level of education of the subject, on the 
specialized skills of the subject as indi- 
cated by technical training, and on the 
interests of the subject as indicated by 
career plans. 

In a few cases we will indicate cor- 


relations with other variables. These cor- 
relations will refer only to problem 2, 
unless otherwise stated. The Examiner's 
Office of the University of Chicago has 
permitted us access to the results of vari- 
ous tests administered to students by that 
office. Among the tests administered 
were the American Council on Educa- 
tion Psychological Examination and a 
mathematical aptitude test. In addition, 
a number of tests were administered 
which have subtests designated as “ana- 
lytic” scales. An average score for the 
combined analytic sections of these tests 
was computed and will be referred to as 
Total Analysis. Some of the correlations 
between these tests and PSI variables 
will be presented in what follows. 

Our concern here, then, is to under- 
stand the set of PSI variables, rather 


А г six 
than the differences between our $ 
groups. 

E. Work Variables 
а e- 
1. Tıme—the number of minutes 1 


quired for solution to be achieved. Solu- 
tion criterion is the ability to light en 
center light using only buttons 4» 5, anc 
6 in some combination or sequence: Уз 
three successive attempts. (Data for this 
variable are presented in Table 1.) 
Correlations. Time correlates 
Effort (see below) .88, with the ACE 
scale —.36, with the ACE “Q” 
—.37, and with Total Analysis a$ 
scribed in the preceding section —-4° 
er than 


with 


scale 
de- 


Discussion. Time is a power index rath yeen 
a process index. Certain relationships Ee ей 
Time and process are, of course, to be RR 0 
since the process must occupy time. Inspectio not 
the data shows that the various groups oe rire 
differ markedly on the time which they req. is 
for solution of problem 1. The differen ^ 
appreciably greater on problem 2. Г dent 
groups 2, 5, and 6, which are the college pis anc 
groups, require less time than groups !, up 
4, Which are the Ph.D. groups. As will b nese 
later, the faster achievement of solution bY e 


Co 
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TABLE r 
== а TIME 
Problem All s о 
a. Group 1 roup2 Group 3 Grou! E 
| cat, EBI бы) Cy es quai quu 
Problem "d college) РЕР) — PD) College) College) 
roblem 2 23.8 22.0 18. 22 
БЕ = 44.5 41.0 д шн 2528 d ч 
ifference | 3 . © 
e | —20.7 —19.0 —12.6 —17.2 —18.8 —16.5 229.6 


grou 


Ps is ace 
accom 1 
1 panied by what might be termed 


esser e 
Of the ула. of inquiry, When the difficulty 
is manifested s "je increased, the same ren i 
non-natural. with the exception of group 6 ihe 
counters b s college group, ud en- 
i» Problem difficulty with various aspects of 
he difficulty, is i increase in required time as 
ae for all зу сы is approximately the 
з that the athe except group 6. Thus it ap- 
пе required f ect of education as such on the 
B slow the pita solution of a PSI problem is 
Al interes 2s E The effect of special techni- 
рн groups 5 мазак in the differential 
“tech 10 cause iiy 4 and groups 5 and 6, ap- 
а" eam y faster performance (where 
ям 5). None of gon to denote natural science 
d. large. The bu differences seems particu- 
is үш ох Ше eme correlation between per- 
TE This кога problems for this variable 
in КУ Since the m tion is not really a reliability 
Yir structu wo problems are not equivalent 
ire. The increased familiarity of 


ле 
subje 
1 jects wi 
does With the PS 
е PSI a r solvi 
no SI after solving problem 


Prob} t brin 

ase S Ап Жо faster performance on 

Чоп w a ate те: i 

ministy ould requir m asure of the inter- 
та c varying the order of ad- 


tion 
» Whi 
ich has not yet been done. 


2. о 
; UESTI 
о 
Ns—the number of ques 


lon. 
5 requi 
ште а x 
th Á Guest d for solution to be achieved. 
We Co 
activa of е тсе the series of manipu- 
first ion of any which occurs between the 
y element in the net and the 


Subs 
Sequent ti 
time period in which all ele- 


tive. In other words, a question is 
i c a manipulation or series of 
manipulations which interact or аге contempo- 
raneous. Both the specific content of each ques- 
tion and the time at which it was asked are 
recorded, (Data for this variable are presented 


in Table 2.) 


ments are inac 
considered to b 


(Q) is related 
to Rate (see 
Redundancy 


Correlations. Questions 
to Effort (see following) -97> 


following) -49> t° Actual 
(see following) 0n problem 1, 72 and on 
roblem 2, 77» ® the ACE “Т” scale 


— .43, to the AGE “О” scale — .35, and 


to Total Analysis —.46. 
ns required 


umber of questio 
so a power 


be achieved is al 
that is, the 


ture of the questions, 

he questions, is a process index, The 
treatment 0 i a will be presented. їп the 
next section, on Information Variables. The cor- 
relation between О and Effort will be easily 
d when thc izes that Effort 
nately М mes T- 
1 Actual Redun 


between Q anc г of 
ill be discussed in detail in the 
i As with time, the 


Discussion. Then 
for the solution to 


importance а 

resentatior of that varia le. 
data suggest that there is а reasonably high re- 
lationship petween the power aspects of PSI per 
formance and А ep” and "Q' scales and that 
the relationship 0 the achievement ofa produce 
on the PSI to the ability involved in the “Analy- 


3888 ‘ABLE 2 
Group 4 Group 5 ШОО 


Probl E 
ED All G Group 2 Group 3 : 
i n ; N Other (Nat. Sci 
P Groups (P nD) Eee) Ooh ‘i SD: College) College) 
roblem 
~ ro I : ЕЕ 
blem 2 бо.т 35.0 51.0 23-0 40:5 ые iia 
Dig, ES 50.0 77-0 41.0 59 
EE —13.0 —24.5 —78.8 
ie: و‎ —26.0 —18.0 3: 
oa 
i Bureau Ednl. 
pun d 
by 
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TABLE 3 
CoMPLEXITY aes 
" E LOG 95 2 - m 6 
2 G Groups; Group 
bl АП Groups CODZ Qu Coup (Nat. Sc. (Other 
Problem Groups (Ph.D) Soleo PhD) PhD) College) College) 
Problem 1 1.71 1.67 1.80 1.53 1.76 
Problem 2 2.30 2.08 2.70 1.08 3, 15 
Difference | .59 дӣ .90 45 —.39 


sis” subtests mentioned above is definite, being 
significant at the .o1 level. The correlation for 
all subjects between performance on problems 1 
and 2 is .46, the same as for Time. Note that 
groups 1, 3, and 4 require fewer questions than 
groups 2, 5, and 6 on both problems, This indi- 
cates that the more educated subjects ask fewer 
questions than the less educated ones. Note also 
that group 3 requires fewer Q than group 4, and 
group 5 requires fewer O than group 6. Thus it 
appears that PSI performance using Q as an index 
is more economical as the educational level of 
the subject goes up, and, if the educational level 
is held constant, training or interest in the na- 
tural sciences is related to more economical per- 
formance. Finally, if we look at the increment in 
Q which accompanies an increment in the diffi- 
culty of the problem, we sce that groups 1, 3, 
and 4 are affected less by this change than the 
other groups, although group 6 is much more 
troubled by this increase in difficulty with re- 
spect to this index than any other group. This 
index then is definitely not independent of edu- 
cation, or specialized training or interests, Fur- 
thermore, the increase in familiarity from the 
first to the second problem is not enough to off- 
set the apparent slight increase in complexity. 


3. COMPLEXITY—the total number of 
manipulations required for solution to 
be achieved, divided by the total number 
of questions (О). This is the average 
complexity over the process as a whole. 
Complexity is also computed question by 
question (discussed under Analytic- 
Synthetic Shift below). (Data for this var- 
iable are presented in Table 3.) 

Correlations. None computed, 

Discussion. Complexity is an index which has 
more of the aspects of а process measure than 
the two indices so far discussed, Characteristi- 
cally, during a performance questions require 
longer and longer series of manipulations, but 


are generally single manipulations at the begin- 
ning of the performance. The differential com- 


plexity of different phases of the process yd in 
às an index of the mode of approach а гае 
process evolves (see section on Approach У 
bles following). Here we ате dealing with Bu 
all complexity, which is an average value es 
ing little of the information inherent 2n iie 
measure itself as a process index. Esami og Pi 
data we sce that on the simpler problem ed 
little variation in the mean complexity Ый 
group to group. As the difficulty of the Peter 
increases, however, we sce that the groups yr icd 
entiate, with groups 2 and б showing an Ир hee 
bly greater increase їп complexity than шко A 
groups. The differential between groups z sf the 
and groups 5 and 6 suggests that the great’ в), 
educational level the less the complexity (1 V$. 


and the more familiarity or interest in 


ıe na 
6). 
tural sciences the less the complexity (5 e 
The extent to which the factors of edum а 
level and specialized training or intercst ity о 
this index appears to depend on the sever! the 
the demands made on the individual a does 
problem. Increased familiarity with the E x in 
not cause a decrease in the complexity „ш 
this case, 
x ; pe" 
4. Rare—the number of questions 


quired for solution divided by the 1 п 
(Data for this variable are presented 
Table 4). oh EF 
Correlations, Rate correlates aiti Re’ 
fort (see following) .37, with Actual vith 
dundancy 57, with Questions 49 Mi 
Pauses (see following) —.80, ш 
ture of Modes (see following) En 1 
Predominant Mode (see following) ^" 


| | ;ariable 
Discussion. Rate is the first PSI SE seid 


have discussed so far which is a process © shows 
Examination of the data for this {йө}, i and 
clearly that there is an underlying integ"! which 
unity to the process of problem-solving o quct 
culminates in (he achievement of а 1 and 
(correlation coefficient between problem 
problem э i; r= 84), yet varies great 
individua] to individual. 


we 


ple- 
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TABLE 4 
КАТЕ 
Group 2 Group 3 Group 4 Group 5 Group 6 
Problem Mis " Group (ast yr. (Nat. Sci. (Other (Nat. Sci. (Other 
m sroups (Ph.D) College) Ph.D.) Ph.D.) College) College) 
Proble E Y = = 
Pope 1 3.01 $42 1.27 2.25 2.62 4.03 
xam 2.27 2.62 1.17 1.70 1.04 3-10 
pup ‚Зо ‚бо до .68 .93 
ص‎ 

If | »-" n" 
аше plots а graph of cumulative questions approximated by a straight line, deviations from 
Out x time, which we will subsequently term the this straight occur in two ways. Occasionally 
le rs ‘raph, the curve so gener ted is charac- one observes inflection points at which a change 
innare a straight line, the slope of which in slope occurs (sce Inflection Points following). 
such simites Rate, The regularity with which and occasionally one observes pauses In output 
the Ps Straight line is observed suggests that in followed by a transient period of accelerated out- 
[ed I situation an individual evaluates the i put until the curve returns to the basic straight 
5 line, after which the previous slope is resumed 


ation whic 3 
0 соте Ы Which he possesses in such a way as 
о а decision in a characteristic time. 


(sce Pat 


While or 


1 ar 
the whole the performances can be 


Output 


following). 
phs are presented. 


In Fig. 4, a number of 
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tforman "ame problem. Notice ngen blems of increasing difficulty. 
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The three top graphs in Fig. 4 represent three 
different persons solving the same problem. 
The top left curve represents an average per- 
formance; while it is approximately a straight 
line, there are a number of minor deviations 
from linearity. The top middle graph repre- 
sents a highly linear performance, obtained 
fairly often (see Changes of Set following). The 
top right graph represents an unusual perform- 
ance with multiple inflection points. The bottom 
three graphs represent the performance of the 
same individual on three problems of increasing 
difficulty. Note the similarity in slope despite the 
increase in difficulty. 

From the correlations above we sce that as 
Rate increases, the economy of Effort and Ques- 
tions increases, the inefficiency of handling in- 
formation (Actual Redundancy) increases and the 
frequency of Pauses decreases, The relation of 
Rate to the two Mode indices shows that speed 
of asking questions is essentially independent of 
whether the subject is in the Analytic or Syn- 
thetic phase of the process (see discussion under 
Approach Variables below). 

The data indicate that both level of educa- 
tion and specialized technical training or interest 
are related to performance at lower rates, A 
general slowing accompanies the increase in diffi- 
culty, but is markedly differential. 


5. Pauses—the number of minutes dur- 
ing which no questions were asked, di- 
vided by the total Time required for 
solution. (Data for this variable are pre- 
sented in Table 5.) 
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of actions differs appreciably with respect to 
grouping. As can be seen from the correlation 
of —.80 between Pauses and Rate, the relation- 
ship between these two is high, yet the схепнез 
of spacing of questions is not the determinant 9t 
Rate. Note the large differentials in the above 
data, We see that educational level seems 9 
affect the frequency of pauses, by contrasting the 
data for groups 1, 3, and 4 with those for groups 
2, 5, and 6. By comparing group 3 with qup e 
and group 5 with group 6, we sce that at thi 
same educational level the group with "кести 
cal" training or interest dis a higher ei 
cidence of pauses. This holds true at both levels 


Е is { :ncrease in the 

of difficulty. There is a general increase ag a 
as the problem increases 

frequency of pauses as the | 1 fre- 


difficulty, Note that the higher the initial a 
quency of pauses, the greater the с ج‎ 
frequency which accompanies greater ee 
Thus this index appears to be somewhat is 
pendent on both the educational level A ity 
special skills of subjects, Increased familiar i 
with the PSI does not result in a decree к 
frequency of pauses, The correlation between 
two problems for this index is .76. 


б. INrLECrION Poixrs—the number a 
inflection points in the Output ae 
(Data for this variable are presented j 
Table 6.) A 

Correlations. Inflection Points cor 
late with Effort (see following) -42- 


: g * ility im 
Discussion. Inflection Points are of utility 


5 E нЕ ;hich there 
E " designatin oints in the process at whic! e 
Correlations. Pauses correlates with has pipe ets in the rate at which m. 
: = inter’ 
Effort (see below), —.22, and with Rate process is being generated, Since WC nes a 
—.8о. the Rate as an index of decision-making. repre- 
| И 1 . | i change in the slope of the output graph Ne be- 
Discussion. Pauses is an index which Bives us sents a change in decision-making time. " sted 
an insight into the constancy with which the lieve that the constant Rate which is man raph 
process goes on. It is possible to have two per- by the straight line form of the Output ws 
formances which display the same Rate, one of indicates a constant set on the part of the “the 
which has no pauses and the other of which con- ject. In su is interpretation 97€ — 
H м Я pport of this interp + the 5С 
tains an appreciable number of pauses, because following facts: As will be described in t em 
of the smoothing which enters into the compu- tion of this paper on Information Variables nich 
tation of Rate. In the two cases the distribution can stipulate the point in the process ap 
TABLE 5 
Pauses 
= سے‎ 6 
roup 
All Group; Group 2 Group 3 Grou 4 Group 5 her. 
Problem | Groups (Ph.D. (18 уг. (Nat. Sci. (Other (Nat. Sci. еге 
| College) Ph.D.) Ph.D.) College) 
Problem r -09 20 +05 .26 15 JTA © 
Problem 2 +18 +30 Ке 39 .23 .20 p 
Difference | —.09 i — .06 —.13 = 768 — .08 s 
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TABLE 6 
INFLECTION POINTS = 

Г Group 2 Group 3 Group 4 Group 5 Group 6 

TM | call, Чр багу. (Nat, Sel. (Other (Nat. Sd) (Other 

— OS D.) College) Ph.D.) Ph.D.) College) College) 

Problem т | 1.5 Ec a m 5 А ; 

Problem 2 | -$ 1.4 i ‚т з PES SS 
`2 22 Pes 

Difference | —1.0 —.8 —1.3 “5 E 1.0 1.9 


the necessary and sufficient information (NASI) 


for the solution has been achieved by the sub- 
ject. When inflection points occur in the Output 
Graph, they are most frequently associated with 
the achievement of the NASI. That , when the 
subject achieves the information which is both 
necessary and sufficient to permit him to infer 
the valid solution to the PSI, characteristically 
there will be a change in the Rate, even though 
solution may not be forthcoming for some time. 

Some figures here may give an indication of 
the extent to which changes in Rate can Бе 
attributed to changes in the state of information 
of the subject. In a group of go performances 
which were analyzed, the following observations 
Were made, Seventeen performances had no in- 
flection points. Forty had one inflection point, 
and in gg of these either the NASI or the Ana- 
lytic-Synthetic Shift (A-S Shift, see following) 
was located at the inflection point (82.5%). 
Twenty-three performances had two inflection 
points, making a total of 46 inflection points for 
this group, and the NASI or A-S shift was coin- 
cident with 25 of the inflection points, ог 54-370: 
Ten had three or more inflection points, making 
а total of до inflection points for this group, and 
the NASI or A-S shift was coincident with 15 of 
the inflection points, or 937.595. As the difficulty 
of problems incrcases, the percentage of such 
coincidences goes up. Finally, on 27 of the go 
performances, a pause occurred immediately be- 
fore the A-S shift. These data appear to support 
the conclusion that the stable Rate observed in 
the PSI is an indication, and perhaps а conse- 
quence, of the existence of а constant set. When 
information input to the individual occurs which 
is probable to cause a change in set, a change in 
Rate usually follows. Thus, we interpret Inflec- 


tion Points as an index of changes in set. When 


Such changes occur in conjunction with changes in 
e information 


the adequacy for solution of the i i 
possessed by the subject, we take this as an in- 
dication that the new information is being in- 
Corporated conceptually. When inflection points 
Occur with no corresponding increment in in- 
formation, we take this to indicate either re- 
organization of the body of information acquired 
to this point, or misinterpretation of the mean- 


ing of the most recently acquired items, Subse- 
quent steps in the process indicate which of the 
above is correct, according to their content. We 
score the appropriateness of "changes of set" by 
relating the loci of inflection points to the loci 
of NASI points, in a yes-no dichotomy, as a 
subscore under this heading. 

Note that the data show an appreciable differ- 
ential between the groups for this variable. 
While this differential is clearer on problem 2 
than on problem 1, it indicates in general that 
the level of education and the extent of special 
technical training or interest are both related to 
frequency of inflection points. Comparing the 
more highly educated groups (1, 3, and 4) to the 
less educated (2, 5, and 6) shows the first differ- 
ential. Holding the educational level constant 
and looking at the effect of special technical 
training or interest by comparing natural scien- 
tists (group 3) with non-natural scientists (group 
4) at the Ph.D. level and similarly (group 5 vs. 
group 6) at the college level, shows the second 
differential. Note also the much smaller increase 
in incidence of inflection points among the more 
educated groups (1, 3, and 4) as the level of 
difficulty of the problem increases. The correla- 
tion between performance on problem 1 and on 
ə for this variable is .51. 


problem 2 

PERCENTAGE OF NONLINEARITY—a 
measure of the extent to which an in- 
flection point changes the subsequent 
performance. If there is no inflection 
point, % NL is zero. If there is an in- 
flection point, a straight line is drawn as 
a line of “best-fit” to the first part of the 
Output Graph. This line is then ex- 
tended to the time of solution. (There is 
sometimes a transient rapid rate for the 
first two or three minutes of perform- 
ance, followed by a steady slower rate. 
If the first straight line is derived from 
the first three minutes of the process 
alone, disregard it and extend the slope 
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TABLE 7 
PERCENTAGE OF NONLID 


EARITY 


Problem All Groups 
Problem т 12.6% 
Problem 2 22.5% 


of the performance after the third min- 
ute.) The area of the large triangle under 
this straight line is determined, as is the 
area between the extended straight line 
and the actual performance polygon, in- 
dicated as the small triangle labeled 
“deviation from linearity” in Fig. 5. The 
ratio of the latter to the former is per- 
centage of Nonlinearity. (Data for this 
variable are presented in Table 7.) 

Correlations. Percentage of Nonline- 
arity correlates with Actual Redundancy 
(see following) .24. 


Discussion, This index is used in conjunction 
with the previous one in order to estimate the 
extent to which an inflection point affects the 
subsequent process, We argue that if a change 
in set is major, the characteristic decision-making 
time which appears to be related to set or con- 
ceptual framework (sce previous discussion) 
should change appreciably. This index measures 
the accumulated amount of change in decision- 
making time summed over the entire process 
from the point of change and stated as а per- 
centage. We assume that if there were no change 
in slope of the output graph, the first straight 
line, extended to the time of solution, would 
closely approximate the actual performance, To 
the extent that this performance proceeds at a 
rate different from this initial rate, the area be- 
tween the performance polygon and the ex- 
tended initial slope will increase. By division by 
the total area under the extended initial slope, 
we state this deviation as a percentage, the sign 


of which indicates whether the change in rate 
was an increase or decrease. 

Percentage of Nonlinearity tends to be larger 
on the more difficult problem. Also, as the % 
NL goes up, the performance tends to become 
somewhat more inefficient insofar as utilization 
of information is concerned. The corrclation be- 
tween performance on this variable from prob- 
lem 1 to problem 2 is .o2. Further inves igation 
with parallel forms of PSI problems, equated fn 
difficulty, will be necessary before one can dE 
termine whether the low correlation indicates 
unreliability of this index, or results from the 
generally less efficient performance which sub- 
jects display on the more difficult problem. 


8. Errorr—the area under the per 
formance polygon on the Output 
Graph. (Data for this variable are pre 
sented in Table 8.) 

Correlations. Effort A 
Pauses —.22, with Questions -97 with 
Rate .37, with Actual Redundancy (5©© 
following) .72, with Time .88, with In 
flection Points .42, with Changes ol ay 
proach (see following) .59, with Relative 
Analytic-Synthetic Shift —.59, with Ab 
solute Inferential Lag .77, with Relative 
Inferential Lag .30, with ACE ep Mee 
5, with ACE “Q” scale — -38 vid 
ACE “L” scale —.36, and with Tota 
Analysis —.51, 


correlates with 


Discussion. Effort was adopted as an jeg i 
attempt to incorporate both Time and Questi ve 
in one measure, As can be seen from the ae y 
correlations, these two measures are sufficien! Y 
highly related so that Effort docs not tcl! 


k -nres coul 
much that either of the simpler measures с 
———— 


= ү ири! 
Effort was measured by plotting the Ov Fire 

Graphs on standard axes and measuring 

pertinent areas by use of an Ott. Planimeter: 


TABLE 8 


EFFORT A 
m^ ы ессе ي‎ ины sepa ي‎ tti 
АП Group 1 Group 2 Group 3 Grou Group § Group 
Problem ^ | Groups (Ph.D) stYr. (Nat. Se. (Other (Nat Sd. (Other, 
College) Ph.D) Ph.D.) College) College) . 
Problem 1 | 379. 155. 209. бя. " s = "3 
Problem 2 815. 408. 462. 321. KA 568. 
Difference —436. —253 —163. —234. =й ms 
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not do as well, Yet, because it did combine the 
two into a single index, a number of correla- 
tions were computed using this index. They can 
be summarized as follows: As effort increases, 
pauses tend to decrease, rate tends to increase, 
changes of set or inflection points become more 
frequent, shift from analysis to synthesis and 
back to analysis occurs more often, inferences 
are less validly drawn and tested, and the shift 
from analysis to synthesis tends to occur rela- 
tively carlier in the proces The correlation be- 
tween performance on the two problems for this 
ure is .42. 

mination of the data above indicates that 
this measure is dependent both on educational 
level and special technical training, as must be 
expected to be consistent with the dependence of 
the two components of this index on the same 
factors, Note the surprising uniformity of the 
increment in Effort with increased difficulty of 
the problem for the various groups except group 
б. 


Note that both ACE "Q" and “L” scales cor- 
relate significantly at the .o5 level with Effort, 
Which indicates that verbal as well as quantita- 
tive skills are useful in PSI performance, yet 
clearly neither is sufficient to be the major de- 
terminant of performance quality. Finally, the 
high correlation, significant at the .o1 level, with 
Total Analysis indicates that the skills involved 
in PSI performance are similar to the kills in- 
volved in the elucidation and application of 
Principles in other areas more relevant IO veal 
life problems, 


F, Information Variables 

Ate 
Process 
situati 


ach step in the problem-solving 
‚ as we observe it in this special 
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taking 
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deration the implications of all pre 
acquired knowledge. This wil be re- 


“red to as Q, 
Ww 
© cin = u 
аар, " Pépresent the process of ac 
Siti x à 
9n of information from the ар 


Parat 
's by an individual as the sum yield 


men of questions, each i-r 
г làn; in its answer an explicit (Qe) 
Yon, ooo (Qi) amount of informa 

is clear that under certain com 


ditions Q; will be a function not only of 
the question just answered, but of in- 
formation previously obtained. Thus, 
the total information content of the an- 
swer to a particular question may not be 
an invariant characteristic of the ques- 
tion, but a function of the sequence of 
questions asked previously. The process 
then can be represented as follows: 


Total information available after the nth 
question = 
z(Q. 4 


Q3 4 Qe+ Qe tee 4 


n 
(Qe + Q)«a + (Qe + Qn = (0, + Q), = 
Jat 

the sum of all explicit and inferred information. 

If the explicit information content of 
the answer to any question, (Q.),. is con- 
tained in the sum of the explicit and 
implicit information content from the 
first answer to the n— first answer elicited 
from the apparatus in the process, we 
will term that question inferrable. Ob- 
taining the answer to that question will 
not further increase the sum of elicited 
possible, by changing 


information, It 15 | 
en set of questions, 


- P iv 
the ordering of а 8 questi 
to vary greatly the number of inferrable 
the set. If we define the 
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Lowe he number of in- 


actual redundancy as t € 
[errable questions in à € ead 
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choice of questions asked in the process, 
treating the information content of the 
answer as only a function of the particu- 
lar question. In order to state the effi- 
ciency of a given process, that is, in order 
to state the extent to which the informa- 
tion which was acquired up to a given 
step was used to determine the decision 
as to what should be the next step in 
the process, we must recognize that the 
rate of acquisition of information may 
well depend on the ordering of questions 
in addition to the nature of the ques- 
tions. 

Further, we reach the conclusion that 
there may be a fortuitous factor involved 
to some extent in the problem-solving 
proces. Although there may be no 
a priori basis for asking a particular 
question first in a series, yet the selection 
of that question as a starting point, or the 
asking of that question before some other 
question but after others, will affect the 
extent to which portions of the informa- 
tion content of the total pool can be 
inferred at any point in the subsequent 
process. (Note: This would appear to 
have interesting consequences in a num- 
ber of areas. It would seem that to Score 
an examination or a psychological test, 
in which the contribution of this kind 
of factor to the performance has not 
been ruled out, on the basis of the 
amount of time taken or any similar 
nonsequential "power" criterion for 
quality of performance, is to include the 
possibility that a subject with a poor 

score is actually more efficient than a 
subject with a "good" score. Considera- 
tion of the residual variance in actual 
redundancy other than that due to the 
number of questions [discussed below] 
shows that ordering is as important as 
what is asked.) 

In order to partial out the contribu- 
tion of this fortuitous factor of initial 


choice, and in order to enable the precise 
evaluation of redundancy by accounting 
for the dependence of information con- 
tent on ordering, we have analyzed each 
problem used in this study in the follow- 
ing way: A flow sheet has been developed 
which shows, for any possible series of 
questions, what the explicit and implicit 
information obtained at each step will 
be. The necessary and sufficient informa- 
tion for the solution of each problem 
and the total content of the information 
pool for each problem is indicated on 
each flow sheet. Detailed exposition of 
such a flow sheet for a typical problem 
requires too much space to be here pre- 
sented. Suffice it to say that when such a 
flow sheet has been constructed, a typical 
problem-solving performance can be we 
quentially analyzed for the efficiency О 
information handling in a few minutes 
with absolute reproducibility and pre- 
cision. . 

The analysis also indicates the point 
in the performance where the necessary 
and sufficient items of information for 
the solution of the problems have been 
explicitly achieved, subsequently ean 
to as the NASI(exp), and the point ow 
these could be implicitly achieved ! 
maximum inference were validly ee 
subsequently referred to as the NAS 
(imp). 

Other measures of redundancy сап be 
defined. There are a number of differen’ 
kinds of redundancy, for example, p 
number of times in a particular perfor” 
ance that the same question is wpe 
We have selected this particular пес 
as best suited to our present purpos 
but recognize that redundancy can mean- 
ingfully be defined in a number | 
ternative ways, and call to the pe 
Of the reader such potentially Use К 
measures of redundancy as repetition a 
particular questions or patterns of qu 
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TABLE 9 
: _ EXHAUSTIVENESS OF INQUIRY 
Problem | АП С Group 2 Group 3 G 
28 топ ;roup 3 roup 4 Grou Gi 

| Groups (Ph.D.) С yr. (Nat. Sci. (Other (Nat. Sá. ра 

ЕЕ | ollege) Ph.D.) Ph.D.) College) College) 
roblem z | 13.6 12.4 13-5 11.8 12.9 13.3 13.9 
e" 13.0 12.8 13.4 12.1 13.4 13.5 12.4 

Jifference & 4 
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prese tems, (Data for this variable ave 
Nted in Table 9) 
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ma Sp ена i distribution. Thus, 
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x, Examination of the cor- 
relation between this measure and Actual Re- 
dundancy (sce following), which is a process in- 
dex, shows that the two are almost. completely 
independent. This is an unexpected finding. One 
would normally expect to find that as stereo- 
typy of response increases, redundancy increases. 
This apparent contradiction is probably due to 
the existence of two sources of low values for 
this measure: first, the incisive performance re- 
quires only a few items to be elicited; second, 
the disorganized performance may be highly 
stereotyped, or may be highly exploratory but 
undirected, Given two persons who required the 
same number of questions for solution, one 
might expect the one with a lower Exhaustive- 
ness MEANT emonstrate ess Желі у com- 


wd 
Y ih formances were 
cepta provided drat bath PE ١ 
n long. The data suggest that educational 
d pecialized training affect this variable 
bers of all 


у. 10150 


than а process inde: 


a priori. 
2. ACTUAL Repunpancy—the total 
number of inferrable questions, as de- 
divided 


ding discussion, 
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(see following) .59, with Absolute Syn- 
thetic Lag (see following) Explicit —.41 
and Implicit .47, with ACE “Q” scale 
—.24, ACE “L” scale —.14, and Total 
Analysis —.46. 

Discussion: Actual Redundancy is a process 
variable, in the sense that it describes the logical 
efficiency of the problem-solving performance as 
a whole by examining it step by step. As pointed 
out in the theoretical discussion in which this 
measure was developed (see Section F, paragraph 
5), опе expects that ordering as well as the actual 
questions asked will determine the rate of acqui- 
sition of information. In support of this, note 
that the correlation between Questions and this 
variable is .72 and .77 for the two problems used. 
This indicates that about 48% of the variance in 
Actual Redundancy on the easier problem, and 
41% on the more difficult problem, must be at- 
tributed to the ordering of the questions asked. 
This gives some idea of the importance of se- 
quential analysis of the sort here used in proper 
evaluation of the quality of a process, 

Further inspection of the correlations pre- 
sented above suggests that fast workers tend to 
be more redundant, Redundancy of this sort is 
slightly related to a tendency to carry out analy- 
sis separate from synthesis, as we sce from the 
correlation with Mixture of Modes. It is rela- 
tively independent of whether the performance 
is predominantly analytic or synthetic, as we scc 
from the correlation of .08 with Predominant 
Mode. The correlation with Frequency of 
Change of Approach shows that Redundancy is 
related to a large amount of shift from synthesis 
back to analysis, and vice versa, As Redundancy 
goes up, the change from the analytic phase to 
the synthetic phase comes relatively earlier in 
the performance; this change occurs more pre- 
maturely in the less redundant workers, with 
respect to the achievement of the information 
needed for solution, as we sce from the correla- 
tions with Relative A-S Shift and Absolute $уп- 
thetic Lag. Finally, as Redundancy increases, the 
lag in inference increases, as seen from the cor- 
relation with Absolute Inferential Lag. 


The correlation of this measure [rom prob- 
lem 1 to problem 2 is .60. А 

Examination of the data above shows that this 
index is somewhat dependent on educational 
level and specialized training or interests of a 
technical sort. The first effect may be seen by 
comparing the more educated groups (1, 3. and 
4) with the other groups. Note, however, ien 
group 5, which is a college group, is as Rd 
as group 4, which is a Ph.D. group, with respec 
to this index. Note the relatively uniform IN 
crement їп Redundancy which all groups ce 
play as the difficulty of the problem increases. А 
te also that the correlation of this index 
with the ACE scales is not high, while there d | 
correlation significant at the .o1 confidence level 
between this index and the derived score for 
Total Analysis on all achievement tests for which 
data are available. 


3. NASI Porxr (Iurricrr)—-the point 1n 
the performance where the subject ob- 
tains the information which is necessary 
and sufficient to enable solution of the 
problem, provided that maximum infer- 
ential use is validly made of the data өп 
hand. This point is stated in two eae 
(а) Absolute—the number of pipe 
asked up to and including the point; ue 
(b) Relative—as a percentage of the a 
number of questions. (Data for these уа! 
ables are presented in Table 11.) 

Correlations. None computed. 


3 а index 
Discussion. This measure serves as ап 


Ў ] hc 
of the end of the first phase of inquiry in e 
problem-solving process as observed Е окей 
PSI. Опсе the NASI (Implicit) has been ac solve 
by the subject, he is theoretically able IS anit? 
the problem, That is, he has had the oP PT nich a 
to learn the required relationships оп “ rou ps 
solution must be based. Note that for all Baber 
this point is achieved at about the same кер" 
of questions in both problems, with the © 


TABLE то 


ACTUAL REDUNDANCY 


SS - = = === б 
rT) 

All Group: Group 2 Group з Group 4 Group 5 C ther 

Problem Groups  (Ph.D.) (18 Yr. (Nat. Sci. (Other (Nat. Sci. College 

College) Ph.D) Ph.D.) College) 
A ‚89 
Problem 1 190 de +83 .70 .79 .04 
Problem 2 -86 -7 -92 -69 86 .83 E i 

e dd C 08 
Difference = OF — .08 — .09 —.o8 ~.07 —.о4 Я 
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TABLE 11 
NASI PoixT (IurLiciT) 


Absolute 
Problem Group 2 Group 3 Group 4 Group 5 Grou 
: m Я д ; ро 
| c м RD (ast vr. (Nat. Sci. (Other (Nat. Sci. (Other 
| üps ыз College) Ph.D.) Ph.D.) College) College) 
Problem 1 14.6 14.2 17.9 12.2 1б.1 12.2 12.8 
Problem 2 15-4 2 18.3 10.6 14.6 II.4 20.3 
Dillerence 8 1.5 —.4 1.6 1.5 8 =7.5 
7 Relative 
Problem т | КТ] -47 41 -56 -39 41 224, 
Problem 2 E -34 ‚26 +40 ‚2 .26 E 
Difference | 13 EK 15 -16 12 -15 .o6 


tion of group 6. The slight increase in difficulty 
involved when the second. problem is presented 
has little consequence on the location of the 
NASI (Implicit), Absolute, There is evidence 
that educational level or special training are 
effective in slightly earlier achievement of the 
NASI (Implicit). i 

The NASI (Implicit) Relative tells us the per- 
centage of the total process required for the 
achievement of the NASI (Implicit). Note that 
as educational level or special technical training 
increases, the relative location of the NASI (Im- 
plicit) is later in the performance. Since we know 
from the absolute location that there is little 
difference in the numerator of the fraction 
Which determines the relative value, the differ- 
ence which we observe here is mostly due to the 
denominator, i.c., total questions in the various 
Performances, Note that the effect of the in- 
Crease in difficulty is to shift the location of the 


NASI (Implicit) point about 13% earlier in the 
performance, except for group 6. 

1. NASI Pott (Expuicir)—the point 
in the performance where the subject 
obtains the information which is neces- 
sary and sufficient to enable solution of 
the problem without the necessity of 
inferring any relationships involved. 
(That is, the relevant items have been 
overtly elicited from the PSI.) This point 
is stated in two ways: (а) Absolute—the 
number of questions asked up to and 
including the point; and (b) Relative— 
as a percentage of the total number of 
questions. (Data [or these variables are 
presented in Table 12.) 


TABLE 12 


NASI Рот (ExrLiciT) 


dni —MF Absolute 
Group 3 Group 4 Group 5 Group 6 
Probie All Group I oa (Nat. Ra. (Other (Nat. Sci. (Other 
Groups (Ph.D) College) Ph.D.) Ph.D.) College) College) 
5 53 59 
Probl 46 31 53 21 4o 
Problem 2 06 38 п а зо 68 по 
Difference —20 ar MM = -5 EHE 38 WES 
SS E ee Е Relative 
کی ر‎ E 
E 2 .84 -04 .90 .88 .8o 
P .8 .02 =н = 
Porem 774 т 68 77 86 73 7 
= ix +0. 15 .o8 
Difference 13 11 228 d d 3 _ 
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Correlations: None computed. 


Discussion. This measure serves as an index 

of the end of the second phase of inquiry in the 
problem-solving process as observed using the 
PSI. We conceive of the first phase, ending with 
NASI (Implicit), as an interval in which basic 
structuring of the problem occurs, At the end 
of the first phase, the individual has elicited the 
information which is required minimally for 
solution to be achieved. Almost invariably, solu- 
tion based on such inferences is not forthcoming. 
Instead, there intervenes a period of further in- 
quiry which culminates in the NASI (Explicit), 
in which the material which might have been 
inferred at the end of the first phase is overtly 
elicited. This second phase we tend to conceptu- 
alize as a reassurance period in which verifica- 
tion takes place of the inferences which can be 
drawn after the first phase. ОЁ course, the fact 
that inferences can be drawn does not mean that 
the subject actually does draw them in the sec- 
ond phase, It might well be that the activity in 
the second phase is frequently required for per- 
ception of relationships to become accurate, Yet 
it is of interest that one frequently observes sub- 
jects who verbalize as they work in the second 
phase, and it becomes apparent that inferences 
are being tested. Further, very seldom does one 
observe a performance in which the subject does 
not go ahead after achieving the NASI (Implicit) 
to achieve it explicitly. Subjects who change to 
synthetic behavior, as defined later, before the 
NASI (Explicit) is achieved, almost invariably 
revert to analysis subsequently until the NASI is 
achieved explicitly. 

Examination of the data shows that a marked 
differential exists on the absolute index between 
our various groups. This differential is such as 
to support the interpretation that both educa- 
tional level and specialized technical training or 
interests contribute to the ease with which the 
subject achieves the NASI explicitly. Since the 
various groups end the first phase with the im- 
plicit achievement of the NASI almost identi- 
cally, this difference in the location of the NASI 
(Explicit) point must be due to the variable 
length of the interval between the two points, 
This will be discussed in the next section, 

Note that groups with higher educational 
levels and greater interest or training in special- 
ized technical areas tend to achieve the NASI 
explicitly relatively later in the over-all per- 
formance. This implies that these groups have 
less difficulty in combining and applying the re- 
sults of analysis than groups with less education 
or technical sophistication. 

Finally, note that the decrement in the Rela- 
tive NASI (Explicit) as the difficulty of the prob- 
lem is increased is almost identical with that for 
NASI (Implicit), with the exception of group 4. 


This means that as the difficulty of the problem 
is increased there is a decrease in the portion J 
the problem which is occupied by the first e 
second phases of analysis. Therefore it follows 
that there must be a compensatory increase к 
the relative portion of the process асаан Ө 
activities other than analysis of this sort, as ps 
culty increases, In other words, the effect o zd 
creasing the difficulty of the problem is ea 
cause relatively more analytic trouble, but та 

to cause relatively more synthetic difficulty. 


5. INFERENTIAL Lac—the number 2 
questions which intervene between a 
achievement of the NASI implicitly anc 
explicitly, The Inferential Lag can = 
stated in two ways: (а) Absolute—t = 
actual number of questions in the ш 
val between the two NASI points; aes: 
(b) Relative—the number of see 
the interval divided by the total num" i 
of questions required for solution. aptos 
for these variables are presented in Ta 
13. 

is Absolute Inferential i 
correlates with Effort .77 and with wp 
Redundancy .59. Relative Inferenua’ ^ 
correlates with Effort .30. 


indices 
Discussion, 'The two Inferential Lag s 
are measures of the quality of the i proc 
drawn by the subject in the course of t - 
ess. They are indicative of the failure of 
ject to infer properly from the data e 
gathered or of his need for reassurance 
inferences which he has made, which he by а 
making the conclusion overt rather a арі 
tempting to utilize the conclusions in A ojt) осе 
solutions. The fact that the NASI ey ae peen 
curs in a performance once the NASI "clusions 
achieved implicitly is evidence that ave P : 
which could have been drawn either 1 n Ш 
been drawn or have not been accepted. gence 
interval, the subject accumulates direct ference 
Which provides the NASI without ы vious 
Since this material is redundant, it 15 ith our 
that this index should correlate well bor E 
redundancy index for the whole perform? in 
it does. It is also obvious that difficulty "d 
second phase of the process, as measure. ex ? 
lag, will correlate significantly with our T7. 
over-all difficulty—Effort, olute UN 
Examination of the data for the pet [o 
dex suggests strongly that both educat? ес, 
and the amount of training or interest! етеп? 
cal fields are related to the size of the 1P 
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TABLE 13 
INFERENTIAL LAG 
Absolute 
Problem Group 2 Group 3 Group 4 Group Group 6 
elt А стор (ist yr. (Nat. Sci. (Other (Nat. Be. (Other 
р и College) Ph.D.) Ph.D.) College) College) 
Problem 1 40.0 17.2 34۰7 9.8 24.0 40.7 46.3 
Problem 2 62.7 25.5 34.0 13-7 36.5 55.5 89.0 
= Difference —22.7 —8.3 0.7 —3.9 —12.$ —14.8 —4$,7 
те Relative 
Problem т .52 45 -44 -38 50 -46 .56 
РгоЫет 2 .5o .48 -40 -37 59 .48 +54 
Difference .02 -.03 .04 .OI —.09 —.02 +02 


lag. Note, however, that the consequence of in- 
creasing difficulty of the problem is to lengthen 
this interval for all groups except the first-year 
college group, group 2. The difference between 
Broups 1, 3, and 4—the more educated groups— 
and the less educated groups is more perceptible 
on the easier problem, The difference between 
groups з and 5 and their less technically sophis- 
licated counterparts at equivalent educational 
levels tends to be more marked on the more diffi- 
cult problem. This suggests that as the com- 
plexity of the task increases, education as such 
is less advantageous than technical sophistica- 
tion, 
Note that the more technically sophisticated 
groups devote relatively less of the over-all PSI 
performance to the inferential lag, while this 
distinction does not appear in favor of the more 
educated group compared with the first-year 
group (1 vs. 9). Finally, let us point out that the 
effect of increasing the difficulty of the task on 
the proportion of the over-all performance spent 
in the inferential lag is yemarkably little. This 
suggests that increase in difficulty is not followed 
by an increase in the relative difficulty of infer- 
ence. However, the correlations between ie 
ance on the two problems are 3e for the pos 
ша index and —1й for the relative vx 
rni, relation is significant at ws m the 
dime indicates that the ellect at WEIS Ri 
ing Ulty of the problem is not the same 0n 

pdividual ат :nted out that 
he EO it should be pointed ой! 
ility, iia pee given do not indicate Taon, 
and one К this is a measure of OTB? opo 
“tion бї as no reason to expect similar qie 
Severity SEM for two tasks which differ A al 
Tasks of the demands made on the individual. 

equated difficulty must be used 


fin gy to ob- 
a ayy +. has 
Not been ome dure of reliability and this h 


a 


G. Approach Variables 
A graph is drawn of the duration of 
the average question in each minute 
plotted versus time, to which we will 
subsequently refer as the Analysis-Syn- 
thesis (A-S) Graph (see Section I). To 
draw this graph, we sum the series of 
manipulations of the apparatus which 
each minute, divide by the num- 
asked in that minute, 
and plot the result versus that minute. 
We assume that the following distinc- 
tion can be made between two modes of 
approach to the problem. Direct ques- 
tions about particular propositions or 
relationships can be asked with very short 
series of manipulations. Such short ma- 
nipulative combinations we term as of 
the analytic mode. When the subject 
attempts to utilize relationships which 
he has analyzed to achieve solution of 
the problem, combining them and caus- 
ing interactions to occur between them, 
ries of m ns composing a 


occur in 
ber of questions 


anipulatio 


the se 
tion becomes longer. Such long ma 
id we term the 


nipulative с 
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related manipulations (steps) per average 
question in that interval, or if there are 
exactly two steps per question in that 
interval and the interval is bracketed on 
either side by intervals with less than 
two steps per question, or if there are 
exactly two steps per question in that 
interval and only one adjacent interval 
has less than two steps per question and 
only one question was asked in the rele- 
vant interval. A time interval is classed 
as synthetic if there are more than two 
steps per question in that interval, or if 
there are exactly two steps per question 
in the interval and it is bracketed by two 
intervals each of which contains more 
than two steps per question, or if there 
are exactly two steps per question and 
only one adjacent interval has more than 
two steps per question and more than 
one question was asked in the relevant 
interval. Note that this analysis could be 
carried out question-by-question instead 
of minute-by-minute. Such an analysis 
would be more precise but much more 
laborious and has not been extensively 
pursued. 

The performance can in this fashion 
be characterized as a period of so many 
minutes of analysis (A) followed by a 
period of so many minutes of synthesis 
(S) followed by . . . etc. Consider a per- 
formance in which five minutes of A is 
followed by three minutes of S is fol- 
lowed by five minutes of A is followed by 
seven minutes of S. We write this as 
54385478. The total time of this per- 
formance is 5 plus 3 plus 5 plus 7, or 20 
minutes. There was a total of 10 minutes 
spent in the analytic mode, or 50%. Ten 
minutes were spent in the synthetic 
mode, or 50%. (Note that if this analysis 
were done on the basis of a question-by- 
question analysis, the straight line phe- 
nomenon described in the section on 

Output Graphs would result in approxi- 


mately the same distribution of analysis 
and synthesis.) й 

It can be shown that, at any point in a 
performance, the following is true: the 
percentage of analytic before that point 
minus the percentage of synthetic before 
that point is equal to the percentage of 
synthetic after that point minus the per 
centage of analytic after that point. We 
are now in a position to define a number 
of variables. 

1. ANALYTIC-SYNTUETIC SHIFT РОТІ 
(MS Sumr)—that point in the perform 
ance where the percentage of Analytic 
before minus the percentage of Synthetic 
before is a maximum for the whole per 
formance. This defines a unique point 
which separates the PSI performance 
into a predominantly analytic phase be- 
fore the point and a predominantly syn? 
thetic phase after the point. ‘This index 
can be stated as (a) Absolute, and (b) 
Relative. (Data for these variables are 
presented in Table 14.) А 

Correlations. The Relative А-5 shift 
correlates with Effort —.59, and with 
Actual Redundancy —.gg. 


Discussion. The 


in the remainder of the section on s 
variables, permits us to measure the exte 
s 


which the PSI performance does con: 
Separate and separable phases of infor! ш be 
gathering and information application. It ame "E 
seen in what follows that such a separatio m- 
possible, and that examination of the тй 
ship between the state of information pont 
by the subject and the transition from US vii 
to application is quite informative. Note e rm- 
the shift comes relatively earlier in the ре! i in- 
ance, both Effort and Redundancy tend 10 
crease, sts that 
Examination of the data above sugges than 
more educated groups tend to shift carlic? ically 
less educated Broups, and that more шп than 
sophisticated groups tend to shift earlier ister 
less sophisticated groups, Note that these € һе 
ences do not hold for the relative location i the 
Shift point, It is exceedingly interesting tha tle 
change in the absolute shift point is 30. 
when the difficulty of the problem 1g 195 


easel 
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\ TABLE 14 
ANALYTIC-SYNTHE 
| __ 3 LY Es SyxTHETIC SHIFT Porst (A-S SHIFT) 
Absolute 
Problem 
All ч Сгоир 2 z 
КЕЕН 1:3 er 
— „йы ш College) Ph.D.) PAD) (Nats Sci. (Other 
eem 1 46.3 26.3 1 UM CENT 
roblem 2 47.0 28.4 a E Ew. 33.7 28.0 ne 
= =} 2s 24.2 12.1 2. 
н Difference = A = mE rris 90.5 
| — LN 38 5.8 I4 =750, —4.0 
s Relative 
Dreblem 1 25 EE" " T 
EE roblem 2 = en pe 12 ate 65 ‚78 
Diferen E "E 38 Е iii +88 E 
TE dis 13 ET .04 15 17 o7 
——À s З d * +07 .28 
It suggests 
a there = Rie thd ки nn. of in- relates .08 with Actual Redundancy and 
eae MERE, “rendor Я he subject to shift 48 with Rate. 
rom inspecti ready or not. However, we note 
hat the HR. of the data for the relative index Discussion. This index is a measure of the 
Brenne a" imn appreciably earlier for all relative amounts of analysis and synthesis of 
quie that ne problem. From this we which а PSI performance is composed. We sce 
E iculty of the 5 consequence of increasing the from the correlation of 08 with Actual Redun- 
: the process ак is to increase the proportion dancy that this index is relatively independent 
rather thai SS which is in the synthetic phase of the efficiency of the performance, in terms of 
Performan E to increase analyt pects of the economy of inquiry: the same holds true of Rate. 
Problem nd That is, as the difficulty of the -rhe data show that оп the casier problem there 
10 be Bab up, about the same effort appears is little relation between educational level and 
ma but ‘a to acquire the necessary informa- the AS ratio. The technically sophisticated 
"е constr Ie HM more difficult. to reconcile groups tend to engage in somewhat less analysis 
iii The corre ub hich this information imposes. relative to synthesis than their counterparts m 
к for the be between the two perform- the same educational levels. On ше ща ifi- 
а ative ing 27 bsolute index is 13, and for the cult problem neither educational level nor spe- 
о СОгдап zat; is 33. This suggests that there is cial technical skill or interest appears to be re- 
ion of the internal coherence of the lated. to the Avg ЛЫ: 
Notice that all groups except group 5 show а 
relative to synthesis 
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TABLE 16 
MIXTURE or MODES 
Grou - Group 6 
Problem AL. Ср Ges quld Cups (шош (Other 
ex Groups (PRD) Glens: PRD) GM College) College) 
Problem 1 -79 .8s .76 -QI -80 -75 “89 
Problem 2 .66 .64 18% 63 :65 76 20583 
Difference | be :21 —.o08 .28 FS — .01 36 


deavor in the process, because of the complexity 
of the constraints which must be reconciled. This 
should be considered in view of the fact that the 
two problems, as described earlier, do not differ 
in the total size of the pool of information items 
which stipulates the problems, but differ in the 
number of coincidences which must be achieved 
using these relationships. The reader may recall 
that the more difficult problem requires the sub- 
ject to achieve four coincidences, the easier only 
three. Presumably, it is to this difference that we 
can attribute the above shift toward synthesis. 

The correlation between the two performances 
for this index is ЛО. Thus, even though there is 
a shift in the internal organization of the proc- 
ess as the difficulty is increased, the way in which 
an individual distributes his efforts between the 
acquisition of information and the utilization of 
that information remains fairly characteristic, in 
the sense that one can relate the new organiza- 
tion to the old. 


3. MIXTURE OF Moprs—the percentage 
of the total number of analytic intervals 
in the performance which is located be- 
fore the shift point minus the percentage 
of the total synthetic intervals in the per- 
formance located before the shift point. 
(Data for this variable are presented in 
Table 16.) 

Correlations: Mixture of Modes cor- 
relates with Actual Redundancy —.24 
and with Rate .08. 


Discussion. 'This index measures the extent to 
which PSI performance can be separated into 
distinct analytic and synthetic phases, It is ap- 
parent from an inspection of the data that these 
two phases exist relatively separate from one 
another in both problems, although the definite- 
ness of the separateness is greater on the more 
simple problem. Factors such as education or 
specialized technical training or interest do not 
appear to relate to the extent of this separation 
in a clear fashion. Note, however, that the defi- 
nite separation decreases more over the more 


educated groups than over their somit a 
the difficulty of the problem increases, M » 
exception of group 6. The meaning of this differ- 
ential is not clear. i 1 

The low correlation of this index with valasi 
Redundancy and with Rate suggests that ue 
have in the Approach Variables access to 3 xm 
of dimensions relatively orthogonal to те 
which span the space of Work Variables and 
formation Variables. s ds to 

In summary, then, this variable айе is ту 
distinguish the intermingling of two — udi 
processes which are combined in PSI per hc w- 
ance, The high values which are obtained, a ie 
ing relatively clean separation of the piace etc 
two phases, indicates that the Analytic-Synt mas 
Shift Point, rather than being an ати, сл 
ceptual device, is а concept which has аке к= 
tional utility, enabling the separation 0 usly 
aspects of process which are not homogencous 
distributed in the performance. the 

The correlation for this variable between the 
two performances js —.39. We note from tion 
data above that the greater the separa Ете 
between the two phases on the casier Р е 
lem, the greater will be the changes аре 
difficulty of the problem is increased. Per easy 
individuals whose performance on Ue tinct 
problem is composed of exceedingly E ond 
Phases are more venturesome on Hie силат 
problem, Although the reason for this pe velar 
relationship is far from obvious, the lys sign 
tion is significant at the .o1 level (negative * 
and all), 


4. FREQUENCY ОЕ CHANGE OF APPROACH 
—the total number of inversions "e 
Synthesis back to analysis which EE 
the performance. (Data for this varta 
are presented in Table 17.) -elates 

Correlations. This index ppm 
with Effort -59, and with Actual Re“ 
dancy .49. 

Discussion, The previous index, em 


ion 
Modes, gave us а measure of the separa’ dis- 
the two phases, but did not enable US 
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TABLE 17 
FREQUENCY OF CHANGE OF APPROACH 


Group 2 Group 3 Group 4 Group 5 Group 6 
Problem el : oe ) (ast yr. (Nat. Sci. (Other (Nat. Sci. (Other 
EOUDS: ent College) Ph.D.) Ph.D.) College) College) 
Problem 1 | 3.6 7 3.3 2.0 3.3 3:4 2.6 
Problem 2 | 7.8 5.7 6.5 7-4 4.8 10.8 
Difference | —4.2 —5.3 2.4 4.5 4-1 I.4 —8.2 
ПЗЕ, | 


criminate between the performance which in- 
terrupted an analytic phase with a prolonged 
synthetic interjection and one which shifted 
back and forth repeatedly. The present index 
tells us which case we are dealing with. It 
measures the number of times that a subject 
attempts to synthesize and feels forced to re- 
Vert to analysis for more information. We can 
see by comparing the correlations with Rce- 
dundancy of this and the previous index that 
mixture of modes per se does not relate to a 
higher redundancy, but that frequent and pre- 
Mature attempts at synthesis as measured by 
frequency of change of approach do so relate. 
_ Neither the frequency itself nor the change 
In frequency with greater difficulty appears to 
be highly related to either educational level or 
Specialized technical training or interest. We 
Note that all groups change more frequently on 
the more difficult problem. 

The correlation between the two perform- 
ances for this index is .04. This may be a con- 
sequence of the much greater spread for this 
Variable on the more difficult. problem. 


5. SYNTHETIC LAG (IMPLicIT)—the Ab- 
Solute A-S Shift Point minus the Abso- 


lute NASI Point (Implicit). This index 
is stated as (a) Absolute and (b) Relative. 
(Data for these variables are presented in 


Table 18.) 
Correlations. Absolute Synthetic Lag 


(Implicit) correlates with Actual Redun- 
dancy .47. 


Discussion. This is a measure of the failure 
of the subject to shift from analysis to syn- 
thesis as soon as the sum of inferences which 
can be validly drawn from the information 
he has gathered is sufficient for solution to occur. 
Negative implicit synthetic lag occurs very 
rarely. 

The most impressive fact about this index 
is its remarkably small change when the diffi- 
culty of the problem is increased. Neither the 
absolute nor the relative versions of this index 
show large changes with difficulty, except for 
group 6 in the relative category. These data 
show that shift tends to occur after a char- 
acteristic interval, independent of the objec- 
tive difficulty presented by the problem, if one 
considers group averages, This interval is shorter 
for the educated groups than for the less edu- 


TABLE 18 


Syntetic LAG (IM 


PLICIT) 


СРЕО n = - ў 
ڪڪ‎ iL Absolute 
ar, Group 4 Group 5 Group 6 
Problem Group : бл, (Quer (Nat. Sci (Ое, 
үг. Nat. e 
айы Bi (ды тш Me 
p 72 зүү ne uw 
robl ў : .6 22.7 G 
Proble z 29.0 6 As 13.6 18 
E 32.4 16.2 ar —6.9 2.3 
~l ifference un ET 1.7 —6.4 
Relative 
P 31 1 Us 
Toblem ү -35 5 E ae : 
Pro lem 5 z 725 i5. = 4 —.o8 2£ 
. ы .0. B 
Difference m o2 —.07 ‚оо 


30 ERWIN ROY JOHN 
TABLE 19 
SYNTHETIC LAG (EXPLICIT) 
E Absolute > 
3 = Group 6 
Group 2 Group з Group 4 Group А 
аас АП Group s (1st ts (Nat. Sci. (Other (Nat, Sci. ae 
Groups (Ph.D) College) "ppp, PhD) College) College) _ 
Problem 1 —1I.0 —4.6 —3.4 —2.6 —6.4 
Problem 2 — 30.0 9.4 4.2 1 17-9 
Difference 19.0 4.8 8 11.5 7.8 
Ч ES Za € o4 
Problem 1 —.14 —.17 —.08 —.1$ —.10 4 ee 
Problem 2 =t} —.23 — .03 —.00 —.30 ml UA 
| re uw 18 
Difference +.03 +.06 —.os —.15 Torna —.o6 dé 


cated groups, and shorter for the technically 
more sophisticated groups than for the others. 
This appears to hold true for both versions of 


the index, but is less clear for the relative 
measure. 


The correlation. between this variable and 
Actual Redundancy, significant at the .o1 level, 
is to be expected. Most of what occurs in this 
interval acts to increase redundancy, 

The correlation between the two perform- 
ances for this variable is .o1 for the Absolute 
version and .09 for the Relative. The author is 
at a loss as to the explanation for this low 
correlation in view of the small group changes, 


б. SYNTHETIC LAG (Ехрілсіт) е Ab- 
solute A-S Shift Point minus the Absolute 
NASI Point (Explicit). This index is 
stated both as (a) Absolute and (0) Rela- 
tive. (Data for these variables are pre- 
sented in Table 19.) 

Correlations. Absolute Synthetic Lag 
(Explicit) correlates with Actual Re- 
dundancy —.41. 


Discussion. This index is a measure of the 
failure of the subject to shift to synthesis as 
soon as the explicit yield from analysis is 
sufficient for solution of the problem to occur, 
The following considerations are relevant to an 
evaluation of this index: 

1. If the A-S shift occurs before the NASI 
Point (Explicit) is achieved, the synthetic lag 
is negative. Such a premature shift may be 
adaptive if the subject does not find it sub- 
sequently necessary to return to analysis for 
further items of information or for the NASI 
(Explicit) before achieving solution of the prob- 


the NAST 
the 


lem, since one then concludes that 
(Implicit) м; the basis of operation. Г, 
subject finds it necessary to return 10 shares 
(inversion) after the shift occurs, the ihe 
shift appears to be inappropriate, indic 
overly flexible performance. 
2. If. the A-S Shift occur: 
is explicitly achieved, the Synthetic Lag 
tive, and its magnitude is an indic tion 0 | 
extent to which further inquiry is requires 
the subject before realization that Щн, 
formation already acquired constitutes ап f 
quate basis for solution of the problem. с OF 
3. Whether the Synthetic Lag is pos ed of 
negative, its absolute magnitude is a me sur ae 
the extent to which shift from analysis OAS! 
thesis is inappropriate, provided that the BN 
(Explicit) appears in the performance. ai 
For all groups, the synthetic lag is nega Сав 
both problems, Very seldom does one ob 1 ii 
performance in which it is positive. The ie year 
to which the shift is premature does noe SP all 
to depend on educational level or ы in 
technical training or interests, The onek t 
the magnitude of the lag which onan Б aset 
level of difficulty of the problem is ООШОТ 
also appears to be independent of these us dif- 
In general, the size of the lag increases * occur 
ficulty increases; that is, shift tends to 
more prematurely, except for group 8 
Most of the above discussion also ар à 
the relative version of the index. Note. 09 ip 3 
that as difficulty increases the lag for н 
becomes relatively smaller, while that for à e- 
4 increases, Similarly, the lag for group Since 


TAM i 6 increases: "^. 
creases while that for group 6 Hec in cach 


reatet 


is 


after the NASI 
is posi- 
f the 
by 
in- 
ade- 


ve on 


he 


Das o 
plies bi 
ever. 


* CEPS y 
technical sophistication of group з and 5 a 
tras 
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and 5, shift relatively even more prematurely on 
the difficult. problem. The correlation of —.41 
between the Absolute Synthetic Lag and Actual 
Redundaney, which is significant at the о: 
level, means that as the lag becomes more 
negative, redundancy becomes larger. 

Probably a small negative lag would be 
associated with a less redundant performance 
than a small positive lag. since a positive lag 
precludes good guessing. One would also expect 
large lags, whether negative or positive, to be 
indicative of higher redunda These state- 
Ments, of course, ave speculative, since we have 
so little data on positive lag. The correlation 
between the two performances is .49 for the 
Absolute index and до for the Relative index. 
The tendency to shift to synthesis before analy- 
sis is completed appears to be fairly consistent 
for a given individual. Both correlations are 
Significant at the о: level. 


Н. Mean Values and Standard Deviations 
for All Variables 

The mean values and standard devia- 
tions obtained on the two problems by 


our full population are summarized in 
Table go. It will be noted that not all 
the variables just defined are presented. 
Some of these variables are used in the 
calculation. of others and for no other 
purpose. The correlation between per- 
[ormance on the two problems is also 
presented for each variable. Table 20 is 
then а summary of the data which has 
been presented under the heading “АП 
Groups." 

Once more let us emphasize that this 
set of data should not be regarded as an 
accurate indication of the consistency of 
PSI performance for the individual. Some 
aspects of PSI behavior are markedly 
changed when problem difficulty in- 
creases, perhaps due to the necessity for 
reorganization of process when the se- 
verity of the demands imposed by the 


TABLE 20 


MEANS, STANDARD 


DEVIATIONS, AND "RELIABILITY" 


r=.207 for р=.05, 7 


' or VARIABLES 


Nu Problem 1 
we Mean SD ne Mean SD 
Work Vi P E 
e eee 23.8 16.8 44.5 21.0 
Questions 69.1 53:4 DR 89.8 
Complexity 1.71 23 2.3 . 69 
Rate 3-01 1.25 2.27 Ez 
Pauses .o9 .16 ‚18 .20 
zi 5 2 Bê 
ages of Set А 1.5 1.2 $5 
centage of Nonlinearity 12.6 gs 16.2 
Effort — 379.2 521.5 733.5 
Information Variables á 
>. e e * 8. b 2.5 
Exhaustiveness of Inquiry 13.6 2 A viia ш? A 
Actual Redundancy -79 E 207 бил s 
Inferential Lag (Absolute) 40.0 45 oA _ i “1 7 E 
Inferential Lag (Relative) -52 2 5 x 
Approach Variables 
Analytic-Synthetic Shift (Abs.) 46.3 39 0, А ы 35 са 
Analytic-Synthetic Shift (Rel.) ‚7° к 3$ ud ae 
Synthetic Lag, Explicit (Abs.) —11.0 am p d n p, 
Synthetic Lag, Explicit (Rel) ah к: Ез =o с: 
Synthetic Lag, Implicit (Abs.) Жк, ee 088 44 p? 
Synthetic Lag, Implicit (Rel.) E 3.58 696 "m NC 
Predominant Mode (А/5) c 219 042 7.8 5.8 
requency of Change of Approach 3- 
Mixture of Modes (A minus 5 be- " 5 = eg B 


fore shift) | 
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task on the subject increases. To deter- 
mine reliability, it will be necessary to 
construct two problems of more com- 
parable structure than the two here 
used, and then to analyze the results 


~<— FIRST PHASE IMPLICIT 
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Fic, 5: Output and Analysis-Synthesis Graph. This illust 
PSI scoring is based. 


obtained from the administration of 
these. While it is important to ascertain 
such data, this was not possible in this 
exploratory work. This shortcoming is 
to be regretted. We gain here, in addi- 
tion to an insight into the effects of in- 
creasing difficulty, only an approximate 
idea of the consistency of an individual's 
behavior on repeated exposure to the 
PSI. In view of the known differences 
between the two problems, we might as- 
sume that reliability figures properly 
obtained should be at least as high as 
the correlations so far presented. 


SYNTHETIC 


INFERENTIAL LAG SYNTHETIC 
LAG 


I. Summary of Discussion of Variables 
Now that we have presented the [ull 
set of variables so far developed for the 
PSI, we may gain an overview of the 
analysis which they enable. In Fig. 5, we 
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Tates the salient features on which 


present a combined Output and Analysis- 
Synthesis Graph, on which a number o! 
the variables defined and discussed i? 
the previous pages are illustrated. Varr 
ables from all three areas of PSI perform 
ance are represented in this illustratio™ 


The set of variables which we have defined 
gives information about a number of ae 
aspects of the problem-solving process. aes 
of these variables appear to be sensitive t° P b 
sonality as well as to “cognitive” factors ee 
fested during the performance. Such variablo 
as inferential lag, synthetic lag, frequency пе 
inversion, and even actual redundancy, tO s 
a few, one would expect to be particular 
ceptible to such factors as the effects 0 


ly 5118" 
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anxiety. A thorough analysis of the extent to 
Which the variables listed above, which demon- 
Strably describe characteristic and important 
Parts of a person's problem-solving behavior, at 
least оп the PSI and perhaps in other tasks, 
Сап serve as indices of personality organiza- 
tion is beyond the scope of this paper. Some 
definitely preliminary results relevant to these 
Considerations will be mentioned in Section IV 
following, together with a brief and also pre- 
liminary survey of the relationship between 
Some aspects of PSI performance and other 
less abstract cognitive tasks. The entire popu- 
lation of this study has been recalled for in- 
tensive psychological evaluation by Mr. Sidney 
Blatt, of the Department of Psychology of the 
University of Chicago, working in conjunction 
With Dr, Morris Stein of the same department. 
These same workers are also currently engaged 
In research to investigate the relationship be- 
tween creativity in research chemists and PST 
Performance, Results from these rescarches will 
eventually be forthcoming from their laboratory. 

On the ba of the work already carricd 
Out, it appears clear that much more is in- 
volved in PSI performance than cognitive fac- 
lors alone, Examination of the extent to which 
Stress will change characteristic performance on 
Some of these variables, the extent to which 
Others are relatively invariant across many con- 
ditions, correlative studies between this and 
Other behavioral measures—all must be used 

efore one can specify the dependence of PSI 
behavior on perceptual, personality, and cog- 
nitive elements. 

There is an obvious inadequacy to the vari- 
ables so far developed, if our goal is to under- 
Stand the manner in which the problem-solving 
Process is generated by the interaction of an 
individual with a problem. Initially, the subject 
— 


"To save printing costs, a discussion of the 


effects on PSI performance of education per se, 
ar discipline, and interest 
e specialized train- 
a performance 


education in a particul 
a particular discipline befor 
ing, as well as development of 
Profile, an evaluation of our set of variables as 
comparative indicators, definition and discus- 
Sion of a utility index, examination of the con- 
Sequences of increased difficulty on our vari- 
ables, and a comparison of PSI performance of 
а small group of college students with achieve- 
ment test scores, college records, and personality 
indices, have been deposited with the American 
Documentation Institute. Order Document No. 
5359 from ADI Auxiliary Publications Project, 
Photoduplication Service, Library of Congre 
Washington 25, D.C., remitting in advance $2.25 
for 35 mm. microfilm or $5.00 for photocopies. 
Make checks payable to Chief, Photoduplication 


Service, Library of Congress. 


has a choice between a large number of pos- 
sible moves, with relatively little basis for dis- 
criminating between these as to utility. After 
the first questions asked of the PSI, the in- 
formation which is forthcoming restructures the 
probabilities of various questions being asked 
next, so that instead of approximately equal 
probabilities for all questions, we now have a 
more probable and a less probable subset. The 
next choice is made, in a logical performance, 
from the more probable subset, with a conse- 
quent alteration of probabilities for all moves 
in accordance with the information derived from 
the two moves so far completed, 

Ideally we would like to be able to state 
why the nth step follows the n—first step, and 
in what way it derives from and relates to the 
previous elements in the process. This requires 
that we attempt to relate the state of the in- 
formation possessed by the subject to the next 
step performed by the subject. We would hope 
ultimately to be able to include in the de- 
scription of state those parameters of personal- 
ity and perception and cognition which will 
affect choices between steps of approximately 
equal probabilities based on information con- 
siderations alone. If this is our goal ultimately 
—to understand the dynamics of the process 
through which each step is generated—we must 
acquire both an adequate description of state 
and a knowledge of the function by which the 
transition. probability to the next step is de- 
termined, We have here stated a problem which 
is recognized today as that of analyzing a 
stochastic process with nonindependent transi- 
tion probabilities. Our goal is to learn the 
function which stipulates the manner in which 
transitions from element to clement of the proc- 
ess take place. It is hoped that the PSI will lend 
itself to this sort of analysis. The present 
author feels, however, that some of the variables 
so far developed represent largely static ab- 
stractions rather than dynamic functions, and 
will require much improvement before we can 


approach the above-mentioned goal. 


III. PSI BEHAVIOR 


A. General Description 

A general description of problem- 
solving behavior as demonstrated on the 
Problem-Solving and Information Appa- 
ratus (PSI) can now be presented. 'This 
qualitative sketch represents a "typical" 
performance, to which numerous excep- 
tions can be found in the data. 


The starting point chosen by the subject 
for his initial attempts to structure the PSI 


34 


situation is usually at the “output” end of the 
problem, and consists of a rapid and generally 
exhaustive investigation of the immediate ante- 
cedents of the output. Very seldom, one en- 
counters an individual whose initial procedure 
is to permute the “input” elements in a Sys- 
tematic fashion. The problems are designed to 
make this approach difficult and tedious, and it 
usually gives way to the more systematic output 
analysis described above. 

As the antecedent conditions for the output 
are determined, the rate of asking questions 
diminishes sharply and a slower and steady 
rate is established and maintained, At this point, 
many subjects commence verbalization which 
continues until solution is achieved, and start 
to take notes about the events which they ob- 
serve, Verbalization, when it occurs, is of great 
utility in aiding understanding of the basi 
for certain aspects of the performance. Pauses 
occur during which notes and the apparatus 
аге scrutinized апа — "self-discussed" aloud. 
These pauses are frequently followed by a 
transient acceleration in rate which persists for 
a time proportional to the length of the pause. 
The rate of reaching decisions appears to be 
very stable and characteristic of the individual, 
and is manifested by a steady linear increase 
in the cumulative question curve as drawn on 
the Output Graph. Some system of notation 
is adopted early in the performance, which 
serves as a shorthand rather than as а con- 
ceptual aid, 

One might represent the progress of an indi- 
vidual through the PSI problems as a series of 
reformulations of the problem, each time focus- 
ing on different aspects of the many relation- 
ships involved, After the immediate antecedents 
of the output are discovered, the problem is 
restructured, and becomes in effect the problem 
of achieving these antecedents. Since there are 
several antecedents, they constitute what are in 
a sense subproblems, These are solved separately 
at first, and the result of the first restructuring 
tends to be concentration on the task of learning 
how to achieve these antecedents by using the 
permitted input elements. However, at this 
phase, the various antecedents are considered 
separately, and their achievement tends to be 
viewed as separate problems. The PSI problems 
used in the experiments reported here are so 
built that simple combination of the conditions 

which will achieve the antecedents separately 
results in an interaction between these condi- 
tions which precludes the solution from being 
achieved. А А 

Recognition that the interaction between the 
simple solutions is such as to introduce а new 
factor into the constraints previously recognized 
as relevant in the problem is the prelude to a 
revision and reformulation of the problem as 
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perceived by the individual. It is at this point 
that most subjects encounter their first sub- 
stantial difficulty, First, one must analyze the 
interaction in order to determine the manner 
by which solution is prevented, This phase a 
the problem is often accompanied by long anc 
amusingly emotional verbalized arguments which 
tend to be syllogistic. Ultimately, the subject d эч 
covers the source of the blockage, and this 
discovery, even in staid adults, is greeted with 
sure and satisfaction. Р 
ry of the aspect of the interaction 
between the separate solutions to the subprob- 
lems initiates the final phase of the process, in 
which the subject devises а way to block the 
interaction, "This consists of analyzing relation- 
ships in order to learn how to bring apane - 
result which will prevent another result Бот 
occurring. This is the most serious source O 
difficulty in completion of analysis. iet 
When information is acquired by the Ln e 
which might be expected to change set—in р 
it affords the opportunity to achieve closure’ в 
one of the aspects of the problem—an piene 
point is often observed in the Output GET 
that is, the rate at which questions are iis 
changes to a new and also steady rate. Hie 
phenomenon is particularly of interest when fae 
achievement of the necessary and SUE 
formation (NASI) for solution of the [О п 
occurs implicitly; that is, when the informat ls 
from which solution can be inferred is in ci 
possession of the subject. One frequently 9 h 
serves a change in rate at this point, cven DOE 
the subject continues analysis until he pne 
these items of information (the NAST) A 
This indicates that the significance of the 1n ore 
mation which was on hand at the point c 
the inflection occurred was not effectively di 
corporated by the subject even though it vere 
cause a change of set. Surprisingly, the сай 
whelming majority of our sample went -— 
to obtain the NASI explicitly before they oblem 
attempt to produce a solution to the pr ends 
by a sustained combinatory approach. One con- 
to conclude that inference once drawn 15 


г ‚ ап 
firmed by direct inquiry instead of PY us 
attempt. to apply the results of inferens ag 
testing validity implicitly. The inferentia rance 
may be an index of the amount of reassu™ ted 
needed before an inferred conclusion is ассер 
and translated into action. ante- 

Although the initial investigation of the 5 sys" 
cedent conditions of the output is usually ed 
tematic, there is generally not a pcm 
methodical tracing-back of these рі раг" 
through the network to the input end. Pied. 
ticular relationship is focused upon and da 3 
and then attention is frequently direct 
another aspect of the problem which 15 inest 


nected with the relationship previously 
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gated. There is a tendency for the completion of 
What might be termed "blocks" of analysis, 
Which relate to the subproblems referred to 
cr. These blocks involve groups of related 
or contingent relationships. However, such block 
analysis is frequently interrupted by one or two 
ventures into unrelated aspects of the problem. 
This occurs particularly during the analysis of 
inhibitory tionships, where the subject must 
learn how to do something in order to prevent 
something else from occurring which prohibits 
solution, As mentioned earlier, this is the great- 
est source of difficulty in the problems. (Perhaps 
the sallies into unrelated areas which occur so 
often during this phase of the problem are to be 
considered as some sort of random attempt to 
avoid continued failure here. Often the unrelated 
actions are such as to elicit phenomena perfectly 
well known to the subject, and give somewhat 
the impression that after encountering frustr 
tion in the analysis of a difficult relation he con- 
soles himself briefly by demonstrating his ability 
to control some aspects of the situation.) 
А Repetition of particular questions and elicita 
tion of the same information in a number of 
different ways occurs to an extent which is 
surprising in a university population. Occasion- 
ally one observes repeated patterns of inquiry 
Which are long and compli ed but which 
appear over and over during the process. The 
over-all efficiency of the population, in terms of 
aximum utilization of the information content 
Of even observed, is remarkably low. Even 
though subjects have decided to minimize ques- 
tions rather than time (the instructions, the 
reader may recall, request. that both be mini- 
mized), the use of inference to avoid ing a 
question is minimal. Rather, inference 1s tested 
by sking the question, Subjects give the im- 
Pression of being unwilling to test an inference 
by applying the results to an attempt to achieve 
Solution, as though they somehow attached more 
importance to such solution attempts than to 
other questions, Willingness to test an inference 
by using it in a synthetic attempt might be con- 
sidered an index of willingness to take risk. 
Separate analytic and synthetic phases 
Clearly distinguished in the great m 4 
PSI performances. Generally, the clearer this 
distinction, the more effective is the perform- 
ance, ‘The shift to synthesis usually occurs pre- 
maturely; there is a tendency to try to make-do 
With insufficient information, Subjects will come 
back to analysis when forced to do so by their 
Mability to ‘produce a solution. The shift to 
Synthesis occurs after surprisingly few questions 
11 many cases. 
Although performance is predominantly ana- 
lytic, the shift point is located carly enough on 
à relative scale to make clear that the necessity 


to reconcile simultaneously three constraints, 
which is the logical requirement imposed in 
the two problems used in this work, is exceed- 
ingly difficult for this population. The synthetic 
aspect of the task contributes almost as much to 
the difficulty as does the analytic component. 
Frequently, subjects were asked to describe 
the way in which their manipulations of the 
PSI contributed to the achievement of the out- 
put. It was observed that when synthesis was 
carried to a successful conclusion without a 
complete analysis preceding it, the subject would 
ulty in repeating his solution (which 
s required) and might have a solu- 
a solu- 


have diff 
was alway 
tion which was not parsimonious, i 
tion which contained irrelevant manipulations. 
In such cases, frequently the subject would 
attribute completely erroneous functions to some 
aspects of the solution combination, When in- 
quiry was pursued further, the subject would 
buttress his rationale by citing the existence of 
relationships which he had observed repeatedly 
during the analysis. On a number of occasions, 
the evidence so adduced was diametrically op- 
posed to reality. Thus one gains the impression 
of a perceptual factor in the PSI situation, which 
enables a subject to see one thing, conclude that 
he saw the opposite, and subsequently experi- 
ence the repeated contradiction of his conclusion 
bv observed phenomena without revising the 
way in which he has structured the situation. 


B. Some Effects of Increasing Difficulty 


becomes more difficult, by 
the addition of a single feed- 
back relationship, а number of changes occur 
in the characteristic process. The time required 
for solution to be achieved almost doubles and 
the number of questions asked becomes 1.6 times 
as great. The variety of these questions actually 
decreases, so that we have a tendency toward 
greater stereotypy of response. This increase in 
stereotypy is such that where an item of informa- 
tion will be elicited about 5.4 times in the average 
the easier problem, it will be 
times on the more difficult 
problem. Not only does redundancy increase, but 
something like "rigidity" increases also, а indi- 
cated by а tendency toward perseveration of 
uscless activity on the more difficult. problem. 
These changes occur in spite of the fact that 
subjects almost invariably take notes. 

The complexity of the average question in- 
creases markedly, even though the relationships 
being analyzed are of the identical complexity 
in both problems. The average rate decreases 
appreciably, and the percentage of time spent 
with no manipulation of the PSI doubles. The 
number of changes of set goes up, and the ampli- 
tude of the effect of these changes increases. 


As the problem 
what is essentially 


performance оп 
elicited about 8.4 
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Actual redundancy increases from 79% to 
86%. The fact that an over-all increase of a 
factor of 0.6 in the total number of questions 
asked is accompanied by only 7% increase in 
redundancy is some indication of the extent to 
which analyzing the ordering of a process gen- 
erates conclusions other than those obtained by 
more static methods of treating data. The abso- 
lute inferential lag increases by about 50%, but 
this lag represents about 50% of the total per- 
formance in both cases. In other words, about 
one-half of the performance is occupied by the 
elicitation of relationships which could be in- 
ferred on the basis of information already ob- 
tained, before the necessary and sufficient in- 
formation (NASI) which enables a solution is 
acquired. Further superfluous questions may, 
of course, be asked after this point, and invari- 
ably are. 

The absolute synthetic lag is negative in both 
problems. The average subject tends to shift 
to synthesis before he obtains the NASI (Ex- 
plicit) but subsequently finds it necessary to 
revert to analysis to obtain this, This premature 
shift is greater on the more difficult problem. 
It is interesting that the shift to synthesis occurs 
in both problems at about the same point after 
the achievement of the NASI (Implicit), al- 
though of course this shift is relatively earlier on 
the more difficult problem. The similarity in 
the absolute locus of the shift appears almost 
as though shift occurs after a fixed and char- 
acteristic amount of analysis "whether the prob- 
lem is ready or not." 

The number of inversions from synthesis back 
to analysis doubles with the increase in difficulty, 
the performance becomes much less predomi- 
nantly analytic, and the mixture of analysis and 
synthesis before the shift point is greater. АП of 
these considerations support the conclusion that 
as the difficulty of the problem is increased, this 
group of subjects experienced relatively greater 
obstacles in that aspect of the task which re- 
quired the synthesis of known relationships into 
an integrated product which met the definition 

of a solution. 


Thus it appears that some aspects of 
the PSI performance are fairly constant 
and characteristic, while others are ap- 
preciably modified by increments in dif- 
ficulty. The relatively invariant aspects 
of the behavior might reflect the intrinsic 
organization of the individual perform- 
ance, and might give an insight into the 
extent to which factors related to per- 
sonality influence the problem-solving 


attempt. The more labile aspects of per- 
formance may be consequences of reor- 
ganizations of process under the impact 
of greater stress. 


IV. APPLICATIONS OF THE PSI 
A. Comparisons Between Groups 


In order to clarify the factors which 
contribute to performance on the vari- 
ous Problem-Solving and Information 
Apparatus (PSI) variables, we have pre- 
sented data collected from small groups 
of different composition with respect to 
educational level and area of specialized 
waining or interest. These data have 
been used to estimate the dependence of 
our variables on these factors throughout 
the earlier sections of this paper. How- 
ever, we may also use this material as a 
basis with which to evaluate the over-all 
contribution of education in general and 
of various kinds to PSI performance as a 
whole. 

A number of questions immediately 
come to mind when one considers the 
nature of the task involved in the PSI. 
Will education as such improve perform- 
ance, and in what ways? Is training 1n 
the natural sciences more effective tha? 
training in other areas, and how is such 
increased effectiveness manifested? Сап 
differences be demonstrated which are 
not dependent on such special skills 0 
logical manipulation as are acquired а 
a university? What aspects of real-lile 
performance are related to the behavior 
elicited by the PSI? Can one distinguish 
different modes of behavior which 216 
concomitants of differences in perso 
ality structure? 

The results which we have obtained 
clarify some of the questions asked above: 
We cannot claim to have provided и 
nitive answers to many, nor did we a 
pect to do so in as frankly exploratory * 
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study as this. Many of our comparisons 
are made with small samples which show 
consistent trends but do not reach sta- 
tistical significance. Yet a clear picture 
begins to emerge from the data. Most of 
all, it becomes apparent that answers to 
these questions can be elicited using this 
technique, and areas which merit addi- 
tional intensive inquiry become high- 
lighted. 


The over-all effects of education, as demon- 
stated by the results of a comparison between 
small groups of first- and eighth-year university 
students across our full set of variables, are to 
improve performance, Yet this improvement is 
So slight as to suggest that much more is in- 
volved than simply education per se. А compari- 
Son of Ph.D.level individuals in the natural 
sciences with Ph.D.level individuals in other 
areas of specialization shows that, on each index 
of performance, the natural-scientist group dif- 
fers from the other group in a direction. of 
Breater effectiveness, This raises two possibilities: 
Fither training in the natural sciences is the 
bs sis for superior performance of а group so 
trained, or perhaps individuals with a particular 
kind of habitual approach well-suited to this 
kind of problem are more likely to enter the 
natura] sciences, 

In an effort to ascertain whether the latter 
factor was operative, a group of college students 
Was divided into two subgroups, one of which 
Intends presently to enter the natural sciences 
for specialized study, and another which does 
not. Comparison of these two groups shows that 
in almost every respect, the differences demon- 
Strable between the two groups at the Ph.D.- 
level already exist between these two groups at 
the college level, before any specialized training. 
It appears that an individual is likely to have 
а predictable approach and general behavior 
On the PSI based on his career interests. If one 
assumes that proficiency in the generic skills 
required in an area increases the probability 
that one will enter that area, it would appear 
that the activities involved їп effective PSI 
Performance are more related to the natural sci- 
ences than to other academic disciplines. The 
college-level group which intends to enter the 
Datura] sciences displays more effective behavior 
in а large number of variables than does the 
non-naturalscientist Ph.D.-level group. , 

If one compares groups with interests in the 
Natural sciences at high and low educational 
levels, one can in effect partial out the contribu- 
tion of differential interests to obtain an approx" 


mation of the effects of natural-science education 
on PSI performance, and similarly with groups 
in other areas of endeavor. The results of such 
an evaluation of the impact of different disci- 
plines on PSI performance show that, while there 
is a definite quantitative increase in effectiveness 
after training, that increase is not as great as 
one would hope, does not span all variables 
measured, and is not accompanied by a qualita- 
tive change in the problem-solving process. That 
is, one concludes that for the present population, 
which is by no means representative, and for the 
PSI, which is a particular kind of problem, 
advanced education serves to improve perform- 
ance along pre-existing lines rather than to 
establish new and more effective problem-solving 
techniques, and the amount of this improve- 
ment is not great. 

The data raise the question of the develop- 
mental factors which contribute to the establish- 
ment of the differences which we observe before 
specialized training. A small number of experi- 
ments, which will not be reported in detail, 
carried out with extremely young (6- to 11-year- 
old) children, has shown that such technically 
naive subjects not only can solve PSI problems, 
but that the quality of some of their performances 
was better than that observed with some sub- 
jects at the Ph.D. level. It would be of great 
interest to determine whether two groups emerge 
in a population of children after a given level 
of experience or education is reached, whether 
such factors as home environment can be related 
to differential PSI performance, ог whether 
perhaps two groups are found in the earliest 
measurements which can be made. 


B. Relation Between PSI, Personality, 
and College Performance 


The PSI would appear to require exercise of 
skills in analysis and synthesis, The results ob- 
tained from cross-disciplinary comparisons sug- 
gest that this is indeed the case, if one assumes 
that the natural sciences require skill in analy- 
sis and synthesis. In an attempt to obtain a more 
direct demonstration of the relation between 
facility with the PSI and facility in the perform- 
ance of real-life tasks which require analysis 
and synthesis, we analyzed the performance of 
16 college students on all comprehensive exam- 
inations taken during their residence at the 
University of Chicago. These examinations have 
subtests which measure the ability of the sub- 
to elucidate and apply principles in the 
f the examination. Examining the relation- 
ship between certain variables of PSI perform- 
ance and scores on all such examination subtests 
in the subject's college career to date, we find a 
between effectiveness on the 


ject 
area о! 


clear correspondence 
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PSI and the ability to do well on examination 
sections which require analysis апа synthesis. 
Based on a trichotomy of high, medium, or low 
performance on the criterion of actual redun- 
dancy on the more difficult problem, a better 
ordering was obtained between relative college 
grades on the one hand, and cumulative analytic 
and synthetic subtest scores on the other, than 
by using either ACE scores or college grades 
alone. 

Finally, preliminary examination of the re- 
lationship between differential PSI performance 
and some aspects of per: onality, carried out on 
the same sample of 16 by Mr. Sidney Blatt 
of the Psychology Department of the University 
of Chicago, shows a clear relationship between 
these factors,” 


М. SUMMARY AND CONCLUSIONS 


A technique has been developed and 
herein described and discussed which per- 
mits the detailed observation of the 
problem-solving process rather than its 
product. The technique involves the use 
of an apparatus which permits the pres- 
entation of many different abstract prob- 
lems of quantifiable information content 
and structure. This apparatus is called 
the Problem-Solving and Information 
Apparatus (PSI), 

A technique of constructing and scor- 
ing problems using the PSI has been de- 
scribed and discussed which enables ob- 
jective measurement of the phases of ac- 
quisition, organization, and application 
of information by an individual during 
the process of achieving a solution to ab- 
stract problems. These aspects of the 
problem-solving process are related to the 
solution in a causal fashion, so that the 

solution is seen to be the outcome of a 
unified and cohesive sequential process. 

Using 59 university students and staff 
members at different educational levels 

and with different areas of specialization, 
the dependence of variables in the areas 
of work habits, information acquisition 
and handling, and consistency and ap- 
propriateness of approach to the prob- 
lem, on such factors as education, speci- 


alized technical training, and career in- 
terest has been indicated. 

To the extent that real-life facility in 
the solution of problems of the generic 
sort represented by the task which is 
posed by the PSI may be inlerred from 
observation of PSI performance, a num- 
ber of conclusions are warranted by the 
data. First, marked differences exist be- 
tween groups of individuals engaged in 
the advanced study of certain disciplines. 
Second, these diflerences exist qualita- 
lively between groups intending to enter 
these disciplines, before specialized study 
has occurred. Third, the effect of edu- 
cation varies according to the discipline 
studied, with performance in the various 
areas of the problem-solving process be- 
ing affected differentially. 

The relationship between PSI perform- 
ance and ability to elucidate and apply 
principles in college courses has been 
demonstrated to be appreciable. It would 
appear that difficulty with such manipu- 
lations constitutes а major and severe 
constraint on the performance. of all 
groups studied. 

The data indicate that training or €x 
perience in certain activities, whether ac 
quired early in life or during the course 
of academic studies, bring about habitua- 
tion of an individual to certain kinds of 
conceptual and organizational processes 
which are consistently displayed in d 
peated PSI performance. These processes 
may be more or less appropriate to e 
requirements imposed by this sort © 
problem. In addition to such habitual 
Processes, some aspects of personality ар” 
pear to be reflected in the problem-solv 
ing process, such as, for example. self- 
confidence, anxiety, and compulsivenes- 
Personality factors as well as cognitive 
factors contribute to the PSI perform 
ance. 


PROBL 


Further investigations of the problem-solving 
Process using this technique are necessary before 
definitive analyses of the factors involved in 
performance can be attempted. Many potential 
applications must await the completion of such 
research, The technique may be useful for the 
development and evaluation of teaching and 
remedial techniques in certain fields, by evalua- 
Поп of the performance of individuals before 
and after training in particular skills. By appro- 
priate construction of parallel forms both con- 
taining and not containing specialized informa- 
tion, it might be possible to evaluate perform- 
ance in certain professional and technical arcas 
In a diagnostic way, so as to ascertain the extent 
to which difficulty in the manipulation of ab- 
Stractions rather than deficiency іп knowledge 
Was the major constraint on the technical com- 
Petence of an individual. The relationship of 
performance. on this t to performance on 
Standard. psychological ment devices must 
of course be determined, but perhaps of equal 
interest would be elucidation of the contribution 
of the kind of behavior here observed to abilities 
and skills used in le situations, The 
method might be useful in evaluating the effec- 
tiveness of chemotherapy or other procedures on 
the ability of disturbed individuals to organize 
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cognitive processes. 

In view of the clear differences which have 
been demonstrated between different groups 
entering college, and the relation of these dif- 
ferences to subsequent choice of carcer, it would 
be particularly interesting to gain an under- 
standing of the factors which contribute to the 
establishment of these differences and the acqui- 
sition of these skills, the age level at which they 
are acquired, and the kinds of experience which 
are most conducive to their acquisition and 
cliective retention. The results of such develop- 
mental studies might enable more appropriate 
preparation for those areas of intellectual en- 
deavor where this sort of skill is of importance, 
by facilitating the construction of generalized 
methods for increasing effectiveness on tasks of 
this generic type. Present academic tri ining does 
not appear to change parameters of effective- 
ness qualitatively to any appreciable extent. 

Finally, the possibility of devising simpler 
tasks for laboratory animals, which permit the 
same sort of sequential observation and analy- 
sis of cognitive behavior, should be investigated. 
Present behavioral indices tend to be product 
measures, and possible data of value may be 
obscured by the failure to construct indices 
which give more insight into process itself. 
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Some Dimensions of Interpersonal Relations in 
Three-Man Airplane Crews 


BENJAMIN FRUCHTER, ROBERT R. BLAKE, AND JANE SRYGLEY MOUTON 
University of Texas 


Construction oF THE CREW INTERACTION 
SCALES 
Introduction 


Q of the interesting characteristics 
of a number of modern technologi- 
Cal devices and systems is that their 
Operation requires the team work of 
Several people. This consideration leads 
to questions concerning the extent to 
Which the interpersonal relations, which 
are part of the basis of organizational 
efficiency, can be measured and predicted. 

The purposes of this study were: (a) to 
Construct a scale for assessing the inter- 
Personal relations of three-man, B-47, 
medium-bomber crews consisting of air- 
craft commander (AC), observer (O), and 
copilot (р); (b) to determine the basic 
Content or factors represented in the 
Items; and (c) to estimate the reliability 
and validity of the resulting homogene- 
OUS scales, 


Rationale for Selecting Items 


Several conceptual approaches to 


8'oup phenomena were considered in 
Selecting items to be included in the 
Mitial form of the instrument. 


Theory and experiments in the field of group 
ор namics (5) which have led to the development 

Concepts such as “group standard" and “group 
EL s "group 


` This research was supported in part by the 
R F under Contract Number AF 18(600)-602, 
Hers ced by the 3305th Research and Develop- 
nt Group (Combat Crew Training Research 
“Doratory), Human Resources Research Center, 
andolph Air Force Base, Texas. 


cohesiveness” were reviewed, and items suitable 
for measuring these dimensions in B-47 crews 
were developed. The approach by Homans (3)— 
which relates the group work system to its social 
system through variables such as liking and 
amount of interaction—also was examined, with 
items being written to measure these aspects of 
group behavior. Recent developments in group 
therapy (7) that emphasize group and individual 
relations between members and the leader, and 
also among members, both from a choice and per- 
ceptual standpoint, and that have resulted in 
experiments on the accuracy of interpersonal per- 
ceptions in small groups were examined; and 
items suggested by developments in this area also 
were included. Scales that had been developed 
for measuring the relationships among B-29 crew 
members were examined; some items from such 
scales were included because of their demon- 
strated value in measuring interpersonal rela- 
tions, Finally, the sociometric literature was sur- 
veyed for additional items that had been found 
useful for evaluating interpersonal relations in a 
variety of work situations. 

Items from each source were adapted for meas- 
uring relations among B-17 crew members. A 
final scale was developed after a tentative scale 
had been devised and evaluated through inter- 
views with B-47 crew members and all items had 
been found acceptable in the sense that they as- 
sess relationships that crew members considered to 
be relevant in evaluating their crew performance. 
On the basis of nontechnical considerations con- 
cerning unsuitability of items for use in measur- 
ing crew relations, pointed out by flying and op- 
erational personnel, many items which otherwise 
appeared useful for scaling interpersonal rela- 
tionships were deleted. 


Description of the Crew Interaction Scale 


The Crew Interaction Scale consisted 
of 44 items concerning a variety of as- 
pects of crew relations. 

The first 22 items required each crew member 
to rank himself and each of the other two 
members on some aspect of each member's crew 

s ——ÓÓ 
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performance. Items included the extent to which 
each dominates the decision making of the crew, 
accepts responsibility for both air and ground 
activities, is agrecable and willing to go along 
with crew decisions, and exhibits self-control 
during periods of stress in the air. 

The next five items dealt with how cach crew 
member relates to the others. They included 
items such as the extent to which each crew 
member sees himself as clashing with the others, 
the degree to which cach member feels he can 
influence the others to change decisions, and 
how much each feels a handicap in doing his 
job because of not obtaining necessary informa- 
tion and instructions from other members, 

The final 17 items called for evaluation by 
its members of the crew as a whole, Items such 
as the degree to which the crew functions on the 
basis of an informal code of procedure and the 
extent to which members see the crew as a well 
organized team were included in this section. 


The Ranking-Rating Technique 


In a preliminary laboratory investiga- 
tion with three-man groups, rankings 
had been found to be satisfactorily con- 
sistent from group to group (1). In addi- 
tion to using the ranking method, it was 
also considered desirable to devise a rat- 
ing scale for determining the relative 
strength of each ranking. The crew mem- 
bers were given the following directions: 

In filling out the following items think of 
the three members of your crew—yourself and 
the two others—and try to characterize how each 
bchaves in terms of the scale provided, Use the 
symbol AC for Aircraft Commander, P for Pilot, 
and О for Observer to identify yourself and the 
other two crew members. Make an effort to put 


the identification symbol of only one crew mem- 
ber in any one box. 


Depending on the characteristics 
judged, two types of scales were used. 
One is a bipolar scale with a neutral 


point in the middle, and the other is an 
all-to-none scale with a 


50-per-cent point 
at the center. In constructing a scale, 
the intervals were kept as uniform as 
possible by using the same or similar 
modifying phrases to define positions 
along the continuum. The following is 
an example of the type of scale used to 
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measure bipolar characteristics: 


Rank the members of your crew for the 
extent to which they accept or reject responsibil- 
ity in both air and ground activities. 


О [] Г 


always rejects extremely often very often 


responsibility 
often neither accepts often accepts 
nor rejects re- responsibility 
sponsibility 


О 0 [] 


very often extremely often always ce] 
responsibility 


epts 


An example of a scale used to measure 
characteristics that can be scaled along 
an all-to-none continuum is: 

Rank the members of your crew for the fre- 


: я пре e per- 
quency with which they make errors in the р 
formance of crew activities in the air. 


Г] О O 


сту n 
always makes extremely often very ойе 


errors 
O a little 
often makes errors a littl 


some of the timc 


[] [] 


n ever makes 
very little almost never never ma 


errors 
for both 


According to the instructions 1 
er place 


kinds of scales, each crew memb = 
his rankings of the three crew ал А 
(including himself) at the appropre i 
positions on the nine-point apt 
Tied ratings were not allowed. a 
crew members objected to being t as 
occasionally to make a discrimination: | 


: anai tric 
was required by the “no ties” res ый 
> i ^ y | " 
tion. The objection was met by allov 


the rater to place more than one p 
within a scale interval and indicating 
rank-ordering within the interval. 

Items Included in the Scale 

s on 
iself 
теш. 


The following are the 44 ite™ 
which each crew member rated É- 
and the other two members of PËS © 


=e x тшшш» M C BN oet Еа 
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INTERPERSONAL RELATIO 


1. Rank the members of your crew for the 
extent to. which they accept or rej sponsibil- 
ity in both air and ground activities. 

з. Rank the members of your crew in terms 
of being agreeable and willing to go along with 
crew decisions in both air and ground activities. 

3. Rank the members of your crew in terms of 
the extent they take the initiative in getting 
things done in both air and ground activities. 

4. Rank the members of your crew for the 
frequency with which they make errors in the 
Performance of crew activities in the ai 
5. Rank the members of your crew for 
extent to which they work cooperatively 
the crew in both air and ground activities. 

6. Rank the members of your crew in terms 
of the amount of leadership they provide during 
ar activitic: 

7. Rank the members of your crew for 
extent to which they keep the crew “on 
ball” in both air and ground activities. 

8. Rank the members of your crew for 
емен to which th actions are well-timed 
"ppropriate to the actions of the other members 
of the crew in air activities. 

9. Rank the members of your crew for the 
extent to which they place crew welfare above 
oe mal considerations in both air and ground 
activities, 

10, To what extent do the members of your 
crew follow the directions and suggestions of 
the others in critical air situations? 

11. Rank the members of your crew for the 
extent to which they put your crew at case and 
make being a member of ‘the w more enjoy- 
able in both air and ground activities. 

Rank the members of your crew for the 
А to which they feel frustrated by being 
а member of this crew. 

13. To what extent do the me 
crew usually agree about what should bi 
in the air? | 
. 14. Rank the members of you 
Competence, in air activities, in t 
Which they are assigned. 

15. Rank the members of your 
of the extent to which they want to do a 
lob in both air and ground activities. 

16. Rank the members of your crew for self- 
Contro] during periods of stress in the air. i 
17. Rank the members of your crew for clarity 
Siving directions both in air and ground 
activities, 
ане Rank the members of jon 
ih 1 action with the crew's over 

both air and ground activities. 
mae Rank the members of your 

nt to which they dominate 
Making of the crew in both air 
activities, 


the 
with 


the 
the 


the 
and 


extent 


mbers of your 
е done 


r crew for their 
he specialty to 
crew in terms 
good 


in 
r crew for their 


all performance 


crew for the 
the decision 
and ground 
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зо. Rank the members of your crew in terms 
of the extent to which they keep reminding the 
crew of regulations and SOP in both air and 
ground activities. 

21. Rank the members of your crew in terms 
of the extent to which they are interested in 
their technical specialty їп both air and ground 
activities, 

22. Rank the members of your crew for the 
extent to which they are consistent (level of per- 
formance same from day to day) in their per- 
formance of activities in the air. 

. Rank the other members of your crew 
in terms of the degree you clash with them in 
crew discussions both in the air and on the 


ground. 

Rank the two other members of your 
crew in terms of your liking for them in both 
air and ground relationships. 

Rank the other two members of your 
in terms of the ease with which you can 
them to change their decisions in both 


5. 
crew 
influence 
air and ground activiti 
аб. Rank the other two members of your 
the readiness with which you would 
to back you up if a misunder- 
> in your crew in ground activities. 
two crew members in 
terms of how often you are handicapped in 
doing your job because he has not given you 
ry information or instructions. in both 
and air activities. 
o what extent does your crew work as а 
well organized team rather than as a collection 
of individuals in both air and ground activities? 
29. To what extent do you [ecl "left out" of 
important decisions that your crew makes in 
both air and ground activities? 


How well satisfied are you with the way 
have worked out in this crew 


crew for 
expect them 
standing aro 

27. Rank the other 


30. 
you, personally, 
in its air activit 
gi. What is your 
with your present crew? 
g2. Rate your crew for its enthusiasm for 
flying in present crew assignments. 
` gg. Rate your crew for the speed with which 
it comes to agreement when decisions have to 


be made in the air. 
gq. Rate your crew 
ness to fly under adverse conditions. 
35. Rate your crew for the extent to which 
it is a well organized team in both air and 


attitude toward remaining 


in terms of its willing- 


ground activities. 
36. Rate your 
ability to perform 
air. 
37 
and ground activities. 
38. Rate your crew 


crew for its confidence in its 
in critical situations in the 


. Rate the morale of your crew in both air 


for the extent to which 


4 


it sticks together in the face of disrupting factors 
in both air and ground activities. 

gg. Rate your crew for the extent to which 
it is a "stickler" for following formal or official 
regulations and SOP in both air and ground 


activities. | Р 
до. Rate your crew for its effectiveness in 
planning together in both air and ground activi- 


ties. 

41. Rate your crew for the extent to which it 
is able to maintain a consistent level of flying 
efficiency. 

42. Rate your crew for the frequency with 
which it makes errors in carrying out its air 
assignments. 

43. Rate your crew for its “cockpit confu- 
sion.” 

44. Rate your crew for the extent to which it 
functions in terms of an informal code of proce- 
dures that it has worked out by itself in its 
air activities. 


SAMPLE 


The Crew Interaction Scale was ad- 
ministered to go male flying officers, who 
composed зо intact B-47 crews in opera- 
tional training. Crews had been organ- 
ized for approximately four to five 
months. All were tested individually for 
an hour. Data collection took place over 
a one-week period since only small num- 
bers of personnel could be made avail- 
able in any one test period. 


ITEM ANALYSIS 
Agreement in Crew Ratings by Items 


Since each member rated either the 
crew as a whole or specifically desig- 
nated members, several ratings of a crew 
and its members were obtained from each 
item. From a statistical point of view the 
means of the several ratings should repre- 
sent a better estimate of the characteristic 
being judged than any of the ratings 
considered separately. To represent use- 
ful estimates it is desirable that inter- 
crew means be differentiated adequately 
relative to the individual ratings on the 
characteristic being judged. Horst (4) 
has developed a procedure which can be 
applied to estimating the extent of agree- 


B. FRUCHTER, R. R. BLAKE, AND JANE S. MOUTON 


ment in ratings for any one crew rela- 
tive to the discrimination between crews. 
If ratings are internally consistent, in the 
sense that the several judgments within 
each crew or by each crew member agree 
perfectly, the Horst coefficient would be 
41.0. As the spread of the ratings around 
the mean of the within-crew ratings in- 
creases, the index approaches zero. Horst 
has demonstrated that the formula is 
related to the Spearman-Brown method 
for estimating reliability and to the 
Kuder-Richardson formula (no. 21) for 
internal consistency. 

Each rating was weighted from zero at 
the “most desirable” end of the scale to 
eight at the “least desirable” end. Agree- 
ment indexes were computed for the fol- 
lowing combined-score ratings. (a) The 
variance of the sum of the ratings of the 
crews on a given item was evaluated 
against the within-crew variance. (b) 
Next, the variance of the sum of the 
ratings of the crews on a given item was 
also evaluated against all the within-crew 
variance exclusive of the self ratings. (©) 
Finally, the variance of the sum of the 
ratings for a crew position on a given 
item was evaluated against the within- 
crew variance of those ratings. 

The Horst agreement coefficients for 
the scores included in scales are show? 
in Tables 5 through 10. The results fo 
all of the items may be summarized 25 
follows: 

The interrater agreement indexes for 
the crew totals on the first 27 items (o 


which the first 22 were based on nine 
ratings and the next 5 on six ratings 
high 


each per CreW) were satisfactorily 
with few exceptions. The results indi- 
cated both an acceptable consensus 
among crew members in rating 006 й а, 
other and a satisfactory discriminatio? 
between crew means. 


ings 
There were only three sets of rating 
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for each crew on the last 17 items. As 
computed by the Horst method, these 
items had a wider range of interrater 
agreement values. Also, from the stand- 
point of the kind of judgments involved, 
items 29, 30, and 31 were different from 
the others, since the respondent was ге- 
quired to evaluate his own relationship 
to the crew rather than to rate the crew 
as a unit. Such items might be suitable 
for differentiating among individuals, 
but they would not be suitable for com- 
puting mean crew ratings. Indexes of 
agreement were therefore not obtained 
for these three items. Mean ratings for 
peus 39 and 44 did not adequately dis- 
criminate among crews. Both refer to the 
extent to which the crew functions in 
accordance with a prescribed rather than 
ап informal group structure. The agree- 
ment values for items 33, 34, and 40, con- 
cerned with speed in reaching crew agree 
шеп, readiness to Пу under adverse con- 
ditions, and crew planning effectiveness, 
Were also unsatisfactorily low. 
боч general conclusions from assessing 
H “Sonsistency of crew ratings by the 
orst method were that crew members 
йо Agree in their judgments on most 
Cms and that there is satisfactory dis- 
crimination among the means of differ- 
Cnt crews, Items were next considered 
from the standpoint of validity. 


Criteri TS 

eria Used for Validation 

Two sets of proficiency rankings were 
Procured from supervisory personnel at 
ны Charles Air Force Base. They were 
3 ranking of crews on a wing basis and a 
anking of crews by squadron, within the 
wing, 
ssembled as an 


nths previously. 
which was 


T 67th Wing had been а 
e onal unit four or five mo 
com Wing Standardization Board, W ; 

Posed of skilled personnel representing each 
the three specialties, had completed the 
"ation of the crews used in the validation 


эр 
T 


eval 


study during the week previous to the test ad- 
ministration, In making its evaluations the Wing 
Standardization Board used a fixed set of cri- 
teria, and, on the basis of the evaluation of 
specific components of individual performance 
and coordination among crew members, assigned 
a ranking for each crew in the wing. 

Crews within each squadron were ranked in 
terms of performance efficiency as judged by 
three squadron commanders, who based their 
rankings on information supplied both by flight 
commanders and the squadron operations officers. 
The two most important considerations in mak- 
ing these ratings were the circular error in navi- 

tion and bombing assignments and the level 
of performance of the Observer. 


Rankings by squadron and wing were 
used to distribute the crews into high, 
middle, and low performance groups to 
be used in validating the Crew Interac- 
tion Scale. The three performance groups 
were determined by combining the stand- 
ings given by the Wing Standardization 
Board with squadron rankings. Each of 
the three sets of rankings was given an 
equal weight and the rankings summed 
into a single score. The 10 crews with 
the highest combined rankings were iden- 
tified as the high group; the next 10 as 
the middle group, and the lowest 10 as 
the low performance group. 


Item Validity 

The validity for each item was deter- 
mined in the following manner. An inte- 

al score value was located as close as 
possible to the median of the distribu- 
tion for each item. The number of item- 
ratings above and below the median 
score for each of the three criterion 
groups was then determined. The follow- 
ing diagram shows how the data were set 


up for an item: 


Performance 
Low Middle High 
" Above 2 5 7 
Median Belew 8 Н 


(Item 1 AC) 
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A chi-square test for the significance 
of the differences between the number 
above and below the median in the 
three criterion groups was made, and 
the contingency correlation coefficient, 
corrected for coarse grouping, was cal- 
culated for those items having significant 
chi squares. 

The results indicated that many items 
are related significantly to the criterion 
groups at, or below, the go-per-cent level 
of confidence. Item validity results will 
be discussed in connection with homo- 
geneous scale validation in a later sec- 
tion. 


CONSTRUCTION OF HOMOGENEOUS KEYS 
Scaling Procedure 


To determine whether the items in the 
Crew Interaction Scale could be classified 
into several independent aspects of crew 
behavior, an analysis of the relationships 
among the items was made to identify 
homogeneous keys. The process was be- 
gun by classifying items into six cate- 
gories, based on the rationale for test 
construction. Items were chosen for in- 
clusion in the first tentative clusters on 
the basis of their agreement indexes, or 
significance levels of validity, or both. 
The following set of symbols was used 
for designating scores: 


a. The subscript ¢ following the item num- 
ber indicates the sum of all ratings made on 
the item (eg. 1, (The number “1” refers, of 
course, to item-numbcr.) 

b. The subscript ¢ following the abbrevia- 
tion for a crew position for items 1 to 27 indi- 
cates the sum of the ratings received by that 
crew position (e.g. 1 AC,). 

c. The subscript s following the abbreviation 
for a crew position for items 1 to 27 indicates 
the self-rating by crew position. 

d. The combination of the abbreviations of 
two crew positions separated by an oblique line 
indicates the rating(s) given by the one crew 
position to the other (c.g. 1 AC/O). 

е. On items 28 to 44 the crew positions’ 
abbreviations following the item number indi- 


cate the ratings given the crew by that crew 
position (e.g., 28 AC). 

There were six tentative clusters iden- 
tified at the beginning ol the scaling 
procedure. They were composed of the 


scores listed below. 


Cluster Scores? 
. Leadership i, g АС, 
‚ Cooperation 


Interpersonal Relations 
. Technical Competence 
;. Morale 

6. Formal Group Structure 


grum S asc 


After these preliminary scales had been 
assembled, scaling was done by Stice and 
Knoell's (6) modification of the Wherry- 
Gaylord (8) procedure for homogeneous 
keying. One advantage of this procedure 
is that it does not require the calcula- 
tion of the item intercorrelations at the 
beginning of the analysis. 

The correlations between the total score for 
each scale and score on cach of the 86 пете 
considered for inclusion in the ales wer 
determined and evaluated, A correction Was 
applied to the correlation obtained between the 
item and the scale score to compensate 0} 
spuriousness due to the inclusion of the 
in the scale score, Items not in a scale, tha 
higher correlations with a scale than items 
were included, were then added; and items ha 
ing a tion with Ihe scale were 
dropped. The process was continued in 
to obtain a stable set of items for cach 
Iteration produced stable sets of items for 
scales. Since the Formal Group Structure 
and Interpersonal Relations scale were 
stable, no further analysis was made 0 

Approximately 26 item scores were ine 
in the scales developed in the manner 
scribed above (see Tables 5. б, 8, 
Additional homogeneous keys compos 
not already included in the c 
then derived. Item intercorrelations У idi- 
mated and placed in matrix form, and two а“ in 
tional tentative clusters. were obtained by "d 
spection, Items were added to cluster e 
clusters were stabilized through the perat 
process, Two additional scales, identified by bh 
titles Crew Unity and Crew Coordination, we 
derived (see Tables 7 and 10). 


item 
t hac 
that 


low cor 


order 
scale. 
four 
scale 
nol 


зс „patio 
*See footnote to Table 1 for an expla"? 


of the symbols. 
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TABLE 1 
ITEM SCORES AND TOTAL SCORES ox THE Six HOMOGENEOUS KEYS 


CORRELATIONS BETWE 


Homogeneous Keys 


a Й Crew 
Technical 1 i Crew | Morale Cooperation Unity 
cS : .eadership 8 nity 
> Competence P Coordination 


| Score” 


.12 .4$ 


we 
RAR 


zi 3 АС, "S 
0 


о 


E 
۰3 
3ء‎ 
a 
.2 
“3 
E 


t» C ON Ow 


2 
> 
e 


10, 
10 O/p 
Ir. AC, 


140 
14 P/O 
ET 

13 0/р 
16, 
бр 
i 
17 O/P 
" /1 


چ 


18 P/O 
T9, 


~ 
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TABLE 1 (Continued) 
Homogeneous Keys 

Score? Technical ; Crew i ег. Crew 
Competence Leadership Coordination Morale Cooperation Unity 
380 —.04 -33 45 .09 23 17 
39 P —.27 34 .09 оо 17 17 
40: 213, .38 20 .43 .36 +75 
AT .22 -33 -36 -63 ‚и 40 
424 -34 -27 +31 -45 zi 249 
420 107 —.02 .07 .07 гт 47 
430 —.29 +3ї -56 .16 107 ло 


a The following is the set of symbols for designating the scores in this table. 


а. The subscript ¢ following the item number indicates the sum of all ratings made on the item 


(e.g., 1). 


b. The subscript ¢ following the abbreviation for a crew position for items 1 to 27 indicates the sum 
of the ratings received by that crew position (e.g., т АС). RAS fie scii: 
с. The subscript s following the abbreviation for a crew position for items 1 to 27 indicates the s! 


rating by crew position. 


d. The combination of the abbreviations of two crew positions separated by an oblique line indicates 
the rating(s) given by the one crew position to the other (e.g., т AC/O). ч ; the 

е. On items 28 to 44 the crew positions’ abbreviations following the item number indicates 
ratings given the crew by that crew position (e.g.,28 AC). 


Item-scale correlations were examined and 
further changes were made in scale content by 
eliminating items having high correlations with 


more than one scale, 

These operations resulted їп the 
stabilization of scales identified as Tech- 
nical Competence, Leadership, Crew 
Coordination, Morale, Cooperation, and 
Crew Unity. The loadings of the item 
total and part-scores on the six scales 
are shown in Table 1. 

The item content of the scales, load- 
ings, agreement indexes, and validity 
significance levels are given in a later 
section entitled “Interpretation and Dis- 


Intercorrelations of Scales 


To determine the degree of interde- 
pendence among the scales, the six scale 
scores for each crew were intercorrelated 
by the product-moment method.’ The 
resulting correlations are shown in Table 
з. An inspection of the coefficients indi- 
cates that the Technical Competence 


f the 


? The items were scored so that the end © os 


scale that was considered to represent the en 
favorable responses was given a weight of a 0 
and the response at the least favorable en’ ht. 
the nine-point scale was given a weight of е6, 


Ў Therefore low scores were considered “goo 
cussion of Homogeneous Scales.” scores. 
TABLE 2 
INTERCORRELATIONS OF THE SIX SCALE Keys (MATRIX В.) 
(№ = зо B-47 Crews) 
. a= 
Key 2 3 4 5 

1. Technical Competence 1$ — +03 -57 -34 АЛ 
е ees 162 66 ‘68 pg 
3. Crew Coordination -63 -49 -08 

4. Morale . “74 o 
5. Cooperation me 
6. Crew Unity 


سس 


у ی‎ sS 
ہہ سس‎ 
س‎ г 
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TABLE 3 


MATRIX R7! 
(Inverse of Matrix R.) 


Key 1 2 3 4 5 

р 2.4786 .3078 1.4564 —2.7437 .2077 

: .3078 2.4408 —.$329 — .8105 — +9336 

3 4 —.5329 2.7647 —2.4754 +3325 

4 - H — .8105 —2.4754 5.8863 —1.6627 
208 7 .0336 +3325 —1.6627 2.5881 


scale has the lowest correlations with 
ш other keys. Crew Unity was omitted 
high Казна» consideration because of its 
А correlation with the Morale key. Of 
йз. two, the Morale key seemed more 
Satisfactory. Scales were positively inter- 
a with the exception of a small 
gative coefficient between Technical 
Competence and Crew Coordination. 
‘ As a further test of independence 
of bumar rap scales, the inverse 
If na. € ш relation matrix was computed. 
Touli LIEN could not be inverted it 
Wide m ое an indication that the scales 
SÉ the быша independent and some 
‘ater Reon would be presumed to be 
R) TOONS of others. The inverse 
Serene | shown in Table 3, gave further 
ous scal to the conclusion that the vari- 
a es (except "Crew Unity ) repre- 
entially independent dimensions. 


Scale Reliability 


E reliability for each $ 
item — by correlating odd 
values, ‘Th. based on the nine-point 5C 
Sr esti ne Spearman-Brown correction 
енир we the reliability of the full- 
Correct scales was also applied, and the 
he E values are shown in Table 4 

теш reliabilities for the Leadership, 

eys Coordination, and Cooperation 

Kore. p. somewhat inflated since part 
о rom {һе same item appeared in 

h the odd and even total scores (eg 


cale was de- 
and even 
ale 


two scores derived from Item 1 appear 
on the Leadership key). 

With but one exception the corrected 
odd-even reliability coefficients are satis- 
factorily high for administrative use as 
separate scale measures, The exception 
is the Leadership key which, with a co- 
efficient of .76, is satisfactory as a sub- 


scale within a battery. 


Scale Validity 

In order to assess the validity of the 
derived homogeneous keys, scale scores 
were correlated against the trichotomized 
performance criterion with results as 
shown in Table 4. 

Since the means on the Leadership key 
for the middle and low groups were close 
together and since they both differed 
considerably from the mean for the high 
group, a biserial validity coefficient was 
also computed. The distributions for the 
two lower groups were combined and 
the resulting distribution contrasted with 
the distribution for the high group. A 
biserial coefficient was computed in the 
same manner for the Crew Coordination 
key. In both cases the biserial correla- 
tions obtained in this way were con- 
siderably higher than the corresponding 
indicating that the two scales 
differentiated the high group from the 
other two, rather than providing a uni- 
form differentiation among the three 


criterion levels. 


triserials, 
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AND DISCUSSION. OF 


INTERPRETATION 


HOMOGENEOUS SCALES 


Technical Competence Key 


Composition and definition, Five of the six 
scores composing the Technical Competence key 
shown in ‘Table 5 represent ratings of the Ob- 
server by other members of the crew or by him- 
self on certain of the items, as indicated, Three 
of the five scores represent total crew ratings of 
the Observer by all three crew members. The 
remaining two were ratings of the Observer by 
the Aircraft Commander, These ratings of the 
Observer refer to aspects of his performance such 
as the frequency of errors. competence in per- 
forming the technical specialty, interest in per- 
forming the specialty, and consistency in per- 
forming the crew role, The crew's ratings of 
Satisfaction with over-all crew performance (8 
total) constitute the sixth valid item in the 
key, 

Validity and interpretation. The key has а 


positive, linear relationship of -+.43 with the 
criterion, with individual items ranging in sig- 
nificance from ло to .20. The findings suggest 
that the greater the competence of the Observer 
in carrying out his assigned function, the more 
effective the crew, as ranked on the criterion. 
The finding that items that evaluate the compe- 
tence of the Observer are valid is understand- 
able, since a large portion of the criterion vari- 
ance is considered to be due to the Observer's 


performance. 


Leadership Key 


Composition and definition. Five of the six 
scores that compose the Leadership key, shown in 
Table 6, are ratings of the aircraft commander. 
The sixth is a rating of the copilot, Items are 
ratings of the aircraft. commander's ability to 
put crew members at case, to make being a 
member of the crew enjoyable, and readiness to 
take initiative. The final item includes ratings 
of both the aircraft commander and copilot for 
the degree of acceptance or rejection of responsi- 


TABLE 
‘TECHNICAL COMP NCE KEY 
Significance 
Item and Interrater Con- [буе of 
t Ме adi Agreement —tingency div 
ating om Description Loading Ag ae aî Validity 
Used Item Descrip (Horst) Coefficient (based on o 
4 [A Rank the members of your crew for 
Otal) — the frequency with which they make 
in the performance of crew ac- 61 .so .50 +20 
s in the air. d з 
I4 
$ л : Rank the members of 
otal) their competence, ir , 
the specialty to which they are as- - ба .87 ло 
Н ‚ .5 
signed. 25s 
2 ” of 
i ле ofl 0 Rank the members of your crew 11 
otal) — terms of the extent to which they [ү 
interested in their technical specta 3 T —а 44 20 
in both air and ground activity: lo 
18 M 
(т Rank the members of your crew ba 
otal) their satisfaction with the cre id ы ds 
over-all performance 1n both air а! m 81 +59 d 
ground activities. ` 54 то 
© E: ` 
ЗАС ` i 
Coto (See 4 above). 
220 -— 
(Те Rank the members of your crew ГӨР 
al) the extent to which they аге SO ant 
ent (level of performance same es E did 
day to day) in their performan NE. cC 
activities in the air- — я ея 
= DE Ф se tit was inappropriate to compute this value 
d as 


а D سس‎ SF 
Since ashes in the Agreement Index column 1 
Score yy) Опе rating is involved in the scor nce: 

as not valid at the „го level of signe? 


ndicate th: 
Dashes in the 


Validity Level column indicate that the 
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TABLE 6 
i LEADERSHIP KEY 
on » Significance 
Item and А Interrater Соп- т ol 
Rating Item Description Loading Agreement tingency Validity 
Used (Horst) Coefficient (hased on x?) 
1 AC Rank the members of your crew for 
(Total) the extent to which they accept or 
reject responsibility in both air and " 
ground activities. .42 . «Е 
24 АС Rank the two other members of your 
(Total) crew in terms of your liking for them 8 
in both air and ground relationships. 59 55 H 924 
rii АС Rank the members of your crew for 
(Total) the extent to which they put your 
crew at ease and make being a mem- 
ber of the crew more enjoyable in 
both air and ground activities. 59 “35 ‚40 Jue 
24 P of АС (See 24 AC above) “®7 — .66 02 
3AC Rank the members of your crew in 
(Total) terms of the extent they take the ini- 
tiative in getting things done in both 
air and ground activities. .52 51 +50 59 
100fP (бее т AC above) .48 = КО +05 
bility. The key seems to contain important as- “halo” of effectiveness for their aircraft COH 


pects of the leadership function, 

Validity and interpretation. Al scores in- 
cluded in the present composition of the key 
have a significant relationship with the criterion. 
The key had a negative relationship with the 
criterion, and a triserial validity coefficient of 
—.45 was obtained for the total score. The mean 
for the high group was well differentiated from 
the means of the other two groups, and a 
biserial correlation of the high group versus a 
combination of the other two yielded a coeffi- 
cient of —.55. 

The negative relationship can be interpreted 
in several different ways. For the most effective 
crews, functional leadership may be based in the 
group as a whole rather than being exercised 
by the Aircraft Commander. Under these cir- 
cumstances the Aircraft Commander would be 
less frequently required to put the crew at ease, 
to take initiative, or to accept responsibility. 
An alternative explanation of the negative cor- 
relation or inversion in mean ratings is that the 
members of the least effective crews felt 
threatened by a psychological test of the kind 
from which these data were drawn. They would 
feel compelled to depict their aircraft com- 
manders as unusually competent. The least effec- 
tive crews then might be less secure in admitting 
their own inadequacies so that they create a 


ете rating’: 
the same in 
ation 


mander by giving him more ext 
The effect on ratings would be s 
cither case. Perhaps both these types ores 
occur. 

A third possible interpretation is that (НО 
crews are most effective in which the pe 
commander does not put crew members a Bie 
does not try to make being a member Me ac- 
crew more enjoyable, and does not readily en 
cept responsibility. A similar finding Лав B-29 
reported by Halpin (2). He found ae their 
crews’ ratings of the "consideration" of ] to 
aircraft commanders were negatively reae in 
the effectiveness of these aircraft commander v 
combat as rated by their superiors. He oe 
to this as “the dilemma of leadership, v 
the more the commander pleases his GE . 
lower he is rated in effectiveness by his superior 
In the case of the Crew Interaction Scale ^ aft 
ership key, the crews that rated their an 
commanders highest for making crew ment 
ship more enjoyable, accepting responsib! er- 
etc., were least effective in terms ОЁ the P 
formance criterion. 


Crew Coordination Key 


The Crew Coor- 


Composition iti : 
p 1 and definition. f ratings 


dination key shown in Table 7 consists О 


e. 


^ 
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TABLE 7 
Crew COORDINATION KEY 
Item ; Р igni 
mand S | Interrater Con- Significance 
Used” Item Description Loading Agreement tingency Level, of 
(Horst) Coefficient Validity 
(based on x?) 
8P / 
Р of AC Rank the members of your crew for 
the extent to which their actions are 
well-timed and appropriate to the ac- 
tions of the other members of the 
crew in air activities. 59 = .64 o5 
43 | Rank your crew for its “cockpit con- 
otal) fusion." +56 = -57 то 
8P 
P (See 8 P of AC above). 48 Bs +65 o5 
8AC = 
AC (Sce 8 P of AC above). 48 ar ES 
8 
3 А : Rank your crew [ог the extent to 
otal) which it sticks together in the face of х 
disrupting factors in both air and E 
ground activities. 45 =, 55 EC 


Br M to which the aircraft commander's 
Drive о т actions меле well-timed and appro- 
"cockpi Ratings by the observer of the crew's 
the uo confusion," and of the extent to which 
Ecos oU together in the face of disrupting 
elemen are included in the key. The common 
tion nt among these items seems to be coordina- 

Р, Within the crew. 
eene and interpretation. This key has an 
terion, q curvilinear relationship with the cri- 
themsely, he most effective crews having given 
the Sekt] the highest, the least effective crews 
est mea lower, and the middle crews the low- 
chotomin ratings. A relationship with the tri- 
relation ed criterion of —.23 and a biserial cor- 
The xc were obtained. | d 
ficant eyed items have validities that are ит 
сигу at or beyond the 10-per-cent level, wi : 
тамаа apparent іп the relationships OF 
relatio nl, scores to the criterion. The curvilinear 
that н т might be explained by assuming 
selves S most effective crews did not rate them- 
о dae coordinated because of their уре 
tion wig organization, as discussed in «ошпес. 
effecti v h the Leadership key; whereas itie E 
because crews did not rate themselves hig! у 

Bum ele coordination was inferior to that a 
since de criterion group. On the other hand, 
Close p e means of the two lower pu 
ous weet the curvilinearity may be fortu is 
hich ith the true relationship being опе ір 
ated р the upper criterion group is differ 

rom the other two. 


ni 


Morale Key 

on and definition. All items in the 
Morale key shown in Table 8 are based on the 
combined ratings of the entire crew, Five items 
are crew-level items, while two others represent 
the sum of the individual ratings by the entire 


crew. 

The i 
Ratings su 
trated, and 


Compositi 


tems seem to fall into two subgroups. 
ch as morale of crew, feeling frus- 
enthusiasm for flying in crew assign- 
ments seem to be directly related to morale. 
Items such as maintaining a consistent level of 
efficiency, giving directions with clarity, and being 
a well organized tcam seem to be related to 
crew integration and to technical competence. 
Although items seem to refer to several aspects 
of crew organization, the common content of 
the items is designated as Morale. 

Validity and interpretation. The total score 
for the key has a bimodal distribution and a 
curvilinear relationship with the performance 
criterion. None of the items has a significant 


degree of validity. 


Cooperation Key 

‘on and definition. The Cooperation 
key, shown in Table 9, refers to the extent to 
which members work as a cooperative unit, are 
ready to accept and adopt crew decisions, and 
feel affected because other crew members have 
not provided them necessary information or 
instruction. Three of the five scores, including 


Compositi 
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TABLE 11 
SECOND-ORDER Factor LOADINGS OF THE FivE KEYS 
Loadings | 
тан т 
Кеу Centroid Rotated 
I II 
т. Technical Competence 44 —.61 | -04 +75 . si 
2. Leadership 77 „36 | ‚84 ‚12 7 
3. Crew Coordination 65 41 iT] E "89 
4. Morale .92 —.22 65 „68 d 
5. Cooperation | .83 .06 .73 до | -69 


and Cooperation keys have loadings on 
both of the second-order dimensions and 
seem to be complex at this level. While 
five homogeneous keys can be identified 
through item-scaling, results from the 
second-order analysis support the con- 
clusion that there are two underlying 
factors operating to determine item re- 
sponses. They deal primarily with judg- 
ments of the interpersonal working rela- 
tions (Crew Coordination and Leader- 
ship keys) and with judgments of the 
technical competence of the Observer 
(Technical Competence key). A graphic 
representation of the relationships is 
shown in Fig. 1. 


а] 


TECHNICAL 
{ COMPETENCE 
MORALE 


COOPERATION 


LEADERSHIP 


CREW 
COORDINATION 
—— M 


0 5 10 


Rotated positions of the five homogeneous 


Fic. 1 
keys on the two second-order factors. 


Reliability and Validity of the Combined 
Scores 
t $ 1 ٤ 
Since the relationships of some of th 
á niat: 1 
keys (Leadership, Crew Coordination, 
and Technical Competence) to me 
terion, as well as to one another, E 
y : 5 іпес 
not linear, a combined score was к 
. 1 i a e 
by an unweighted combination of к 
Э, а ights 
scores, rather than by obtaining Wels à 
h : analysis- 
through multiple correlational analya 
” Я i Р" 
The composite score, obtained by d 
а s ] T 
tracting Crew Coordination and Lead € 
à г tence 
ship from the Technical Compete? " 
(Score = TC — CC — L), yielded a mum 
ial validity coefficient of +.62, as show! 
in Table 19. li 
+ . H H is 5- 
Since the high criterion group 1$ me 
criminated from the other two €: 
than they are from one another, a Der 1 
+ 2 Ё 
correlation was also computed p^ 
yielded a validity coefficient of + E: 
ызы " 
The corrected odd-even reliability of t 
Composite score was +.80. 
rder 
Cooperation had loadings on both second-ord o 
factors, with the higher of its two loadings T 
the first factor. It also had a moderately ae 
tive validity coefficient, In order to dee 
whether the inclusion of the Cooperation the 
with its moderate validity would add 0 ion 
validity of the composite, a second combina’ g: 
of keys was obtained by subtracting the Con 
tion score from the composite score men 
above (ie, Score=TC— CC  L — ©: the 
Was anticipated that adding this key p the 
composite score would lower the validity © 


composite, since the Cooperation key had een 
ings on factors with opposite relationships 


> 


гет 
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RELIABILITY AND VALIDITY OF THE COMPOSITE SCORES 


| nen the criterion, The triserial validity coefficient 
x Ten for the combination of four keys was +.56, and 
f Fe the corresponding biserial coefficient, 4.64. The 
| 5 кү. | Ж corrected odd-even reliability coefficient for the 
|S Aloe combined scores from the four keys was +.92. 
| 5| Ё Based on the various combinations of 
| E P DM scales to form a composite score, the 
| Ö bil la maximum validity coefficient was given 
u н зев сач by the subtraction, without weighting, of 
Е Ales the Crew Coordination and Leadership 
sis scores from the Technical Competence 
E = ~ 
А= " id score. 
| ur SuMMARY AND CONCLUSIONS 
~ LASS The purpose of the study was to de- 
2 MEN Net: velop an improved measure of inter- 
E alse personal relations among aircrew mem- 
|= | a} ee bers. Based on the literature and the 
| &8 results from laboratory experiments de- 
gH 25 signed to answer certain methodological 
уз үү questions, forms of the Crew Interaction 
i ЕТЕ $са1е меге constructed and administered 
ы a ie to U. S. Air Force B-47 crews on two 
= 3 separate occasions. The initial admini- 
ХЕ ою stration, which included intensive inter- 
| 8 т аря views of B-47 crews to determine the 
E а ae ИЕ? suitability of items, served as the basis 
* alas for the revised form. The final revised 
NE a as scale was administered to thirty B-47 
E: uH crews of three members each and pro- 
| г =| 79, vided the data for the statistical analysis. 
11 
ا ا‎ ee Agreement in Rating Items 
| a š 3 The results of the interrater reliability 
analysis indicated that there is agree- 
ы a KA ment among the ratings given to a crew 
й il as a whole and to its members individu- 
yop ally, and that there 15 satisfactory dis- 
Bos CE crimination in the average ratings for 
Ess ` the different crews. The highest internal 
Оби p agreement was obtained for the first 22 
о items, for which nine individual judg- 
g ga ments per item were made. The next 
$8 ] J highest internal agreement was E, for 
ES 85 items 23 through 27, that inclu E six 
2 di individual judgments per item. The least 
нЕ 
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internal agreement was found for items 
28 through 44, in which only three indi- 
vidual judgments were represented. 


Item Validity 


Based on a composite performance cri- 
terion composed from rankings of crews 
by the Wing Standardization Board and 
squadron commanders, item validities 
were computed to provide information 
concerning aspects of crew relations 
significant for the performance criterion. 
Approximately 95 scores with significance 
levels at or below .20 for the trichoto- 
mized criterion were identified. 


Homogeneous Scale Development 


Scales derived from the interrelation- 
ships of the ratings lead to the identifi- 
cation of five dimensions of crew rela- 
tions which were identified by the titles 
Technical Competence, Leadership, Mo- 
rale, Crew Coordination, and Coopera- 
tion. The scales representing these di- 
mensions have satisfactory reliability as 
determined by the odd-even method and 
three of them show significant validity. 
One of the three validly predictive scales 
(Technical Competence) has a positive 
relationship to the criterion. The rela- 
tionship of the other two validly predic- 
tive scales (Leadership and Crew Coor- 
dination) with the criterion is negative. 


Second-Order Analysis of Scale Inter- 
correlations 


To determine the manner in which 
the five homogencous scales were grouped 
at the second-order level of complexity 
the scale intercorrelations were subjected 
to factor analysis. Two centroid factors 
were extracted from the scale intercor- 
relations. The analysis of the interrela- 
tionships indicated that there are two 
underlying factors on which ratings are 
based. ‘One can be identified with the 
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Technical Competence key, which has 
a positive, linear, and valid relationship 
to the criterion. Except for further re- 
finement, it seems to be satisfactory in 
its present. form. The significant ratings 
on this scale were lor the crew member 
called the Observer. 

The other basic dimension concerns 
crew relationships. The Leadership and 
Crew Coordination keys have high load- 
ings on it, For the Leadership key the 
high criterion group was differentiated 
sharply from the two lower groups. On 
the Crew Coordination key Ше tendency 
toward curvilinearity was суеп more 
marked, with the mean for the middle 
group being somewhat lower (represent 
ing more integration) than either ex- 
treme group. 

Composite Scores 
from 


In- 
the 


The composite scores obtained 
the homogencous keys of the Crew 
teraction Scale take into account 2 
Opposite relationships of the basic vad 
mensions with the criterion. А validity 
coefficient of .62 (triserial correlation) 
and .76 (biseri correlation) is obtainet 
from the subtraction (without weighting) 
of Leadership and Crew Coordination 
from Technical Competence. n 

One other key, Cooperation, was % 
sidered for inclusion in a compos” 
score since it had moderate negativ? 
validity. However, the Cooperation "i 


. H 5 : p ` 
had appreciable loading on both 3680 1а- 
rele 


order factors, which have opposite : 
tionships to the criterion. Combining © 
with the other scores raised the reliability 
but lowered the validity of the comp? 
ite, 


Conclusion 


ES " А esent 
Within the framework of the ior jt 

: T 
study the research has demonstrated t 5 
У " 

the direct. assessment by crew membe 


га 
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of one another constitutes ап effective 
means for investigating and measuring 
Interpersonal relations in three-man 
crews, and provides a means for predict- 
Ing the efficiency of crews that have al- 
ready been organized. 

An additional significant finding is that 


three keys, Leadership, Crew Coordina- 
tion and Cooperation, were related nega- 
tively with the criterion. Possible inter- 
pretations for these unexpected results 
were presented in connection with the 
discussion of each of the keys as given 
in the text. 
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PROBLEM 


T PRESENT study was an attempt to 
derive postulates from a theoretical 
Consideration of aggression and age in 
relation to nondirective play therapy 
"ud [s predictions from these pos- 
lated 1o pe predictions were then ps 
Бе esas 4 1c гурез of responses that may 
itom porre in nondirec tive play therapy 
si eg ont ol normal intelligence at 
fion Y wonological age levels and of 

aggressive, intermediate, and aggres- 


rp 

вацте C Perimental foreground of this investi- 

sity, пара? loped at Florida State Univer 

Teified is assec; the theoretical background was 
at Carter. Memorial Hospital, Indian- 


polis; 
the Sel the two segments were then blended at 
of n 1901 of Clini and Applied Psychology 


sional Institute, Rich- 


the Ric 
© Richmond Prof 
it was used by the 


Mo; a 
meee сз study completed, s by t 
State Wit, ann doctoral dissertation at Florida 
Tha А in 1998. 

writers gratefully acknowledge the as- 
1 and the staff 
; School and Eugene Boyce and 
ho kindly 


` "E deed Memorial Hospital, both experi- 
tocols сгаріѕіз skilled in the evaluation of pro- 
the Stan served as judges; James Norton of 
atistical Laboratory, Purdue University, 
Cnerously shared his time and his knowl- 
further this investigation; and V. J. 
s, director, School of Clinical & 
en "sychology, who thoughtfully arranged 
ог author's teaching schedule to provide 


tin 
ne 
Or writing. 


sive personality. It also seemed advisable 
to establish norms of play therapy data 
for possible use in future experiments. 

It was postulated that children would 
manifest their aggression in their verbal 
behavior and, further, that aggression 
would be reduced in older children due 
to the process of socialization. From 
these postulates predictions were made 
regarding children’s speech in а поп- 
directive play therapy setting, The fol- 
lowing hypotheses were formulated and 


examined. 


ive children more gen- 
peech habits than other 
children, but also they are more bullying, asser- 
ssy, and exclamatory in their speech. This 
was investigated by examining the 
following types of verbal behavior: aggr 
statements, threatened playroom rules, е 
of definite dec ions, and exclamations. 

nce the imaginary dialogue of storytelling 
excellent medium for aggressive 
tion, the speech of aggres- 
story units than that 


1. Not only are aggre 
erally aggressive in the 


tive, be 
hypoth 


prov ides an 


expression and 
sive children contains more 


of other children. 
3. Assuming that the aggressive child finds as- 


E тихе experiences rewarding, it was hypothe- 
sized that in his speech he makes more favorable 
statements about himself, evidences more interest 
in the counselor, and makes more attempts to 
cstablish a relationship with the counselor than 
do nonaggressive children. 

4. The relationship of the nonaggressive child 
to the counselor is presumably more one of 
dependency than of interest. Hence, it was hy- 
pothesized that nonaggressive children тоге 


frequently make unfavorable statements about 
themselves, express more indecision and doubt, 
——— 
کش‎ а — 
Ber Reasearch 
i EE 
i we 
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and make more attempts to shift responsibility 
to the counselor than do aggressive children. 

5. It was also hypothesized that nonaggressive 
children have more conventional expressions of 
politeness in their speech than aggressive chil- 
dren. For example, aggressive children were 
expected to utter more “Hey!” and “Darn! 
statements (hypothesis 1 above), while it was 
anticipated that the nonaggressive children 
would make more statements of "Hello," “Yes,” 
and "Mmm," type. 


In accordance with each of these hy- 
potheses, based on amount of aggression, 
children with an intermediate amount of 
aggression should make statements whose 
frequency falls between the aggressive 
and nonaggressive children, 

These statements made about aggres- 
sive and nonaggressive children were hy- 
pothesized upon the amount of aggres- 
sion expected to be manifested in their 
speech during nondirective play therapy. 
Similar hypotheses were made about chil- 
dren at different age levels based upon 
the amount of aggression expected to be 
manifested at those levels, These hy- 
potheses were as follows: 

6. Children's speech 
during their fourth an 
children may be more 


fourth than sixth year, 
not generally find ver 


is especially aggressive 
d sixth years, "Though 
aggressive during their 
Such aggressiveness does 
bal expression as much 
during the fourth year as it does during the 


sixth, when speech habits are more developed. 
As the child continues to advance chronologically 
and, presumably, is subjected to increased sociali- 
zation, the aggressive aspects of his speech are 
tempered, Consequently, at the ninth year verbal 
manifestations of aggression are still high, but 
not as high as they are during the sixth and 
fourth year of life. By the twelfth year the ag. 
gressive qualities of speech are negligible com- 
pared to the earlier years of life, 

7. The younger children are expected to be 
more assertive or dominant than the 12-year-olds 
in their statements. In other words, younger 
children, because of their heightened aggressive- 
ness, are hypothesized to make more of their 
own decisions about their playroom activities 
and attempt to shift fewer responsibilities to 
the therapist than would 12-year-old children. 

8, Four-year-old children, because of their 
greater aggressive diffusion, test the limits of 


the playroom more frequently than older ue 
dren. The 12-year-old children, having been sub- 
jected to the most socialization, do the Ea 
amount of limit exploring. Twelve-year-old chi я 
dren confine their interests to routine questions 
indicative of curiosity or of a desire for intor 
mation, rather than questioning how much Ks 
may get away with in the playroom. I 
ference in the verbalizations to be examinec 
concerns statements of this type: It was hypothe- 
sized that the four-year-old child asks, “you 
can break the windows?" The 12-year-old, ins 
stead, asks such things as, "How come d 
brought me?" or, “Does this work like thus an 
son” inance 

9. The greater aggressiveness ог Дош 
of children four, six, and nine years old resu s 
in attempts to relate to the counselor айе 
dence interest in him, rather than passively ae 
cept him. Presumably the older child waits fi : 
the counselor to make the initial moves in AU 
dicating the nature of the relationship. he ед 
year-old, being less aggressive, is not so Du 
in establishing his position in the plar 
To the younger child the establishment О ШИ. 
place is important. Consequently, younger = isle 
dren are also expected to make more de il- 
Statements about themselves than are older €" 
dren. 


10. Another way the more aggressive p 
can establish the assertive atmosphere he ium 
sumably finds rewarding is through the med! А 
of stories. Hence, as a result of increasing socu it 
zation as well as more mature speech habi eae 
was hypothesized that children at the Er p 
level tell fewer stories in the playroom than are 
children at younger ages. The siy year-olds Sir 
expected to tell the most stories since chose 
speech habits are presumably superior to ® 
of the four-year-olds, lite 

11. The 12-year-old children are more am 
as well as more adept conversationally than olc 
Younger children, Hence, when the 12- аир 
speaks he utters more such phrases as, csl 
bye,” and, "Excuse me," than younger ШШЕ 
Instead of telling aggressive stories the ы» а 
old speaks of his family, school, pets, Est 
Breater extent than younger children. Su 127 
it was hypothesized that the more socialized kes 
year-old does less talking to himself and m2 


nger 
fewer Sound effects as he plays than do you! g 
children, 


METHOD 7 

‘ men 
Subjects for the present experime i 
were selected on the basis of chronolos 


з, . ag 
Cal age, intelligence test score, and 38 
gressiveness. 


VERBAL PRESSION 


Ch ronological Age . 


„То be considered chronologically representa- 
Чуе a child could not vary more than four 
montis from the age of 4, 6, 9, ог 12 years. 
m peus was the dependent variable of this 
talk Gp ea ani who could not play and 
date ۳ ks same time were not suitable candi- 
(od sali experiment, The imperfect physical 
cult for E of the three-year-old makes it diffi- 
as dressi ж) to perform two motor acts, such 
Шр Ден talking, simultancously, The 
acts Sine child can perform two such motor 
at Sr: neous! : It has also been noted that 

years the child ceases the mere manipula- 


tion 3 > 

PA toys and becomes interested іп more 
їйє ис play activities. At this age also 
i ild is said to become susceptible to flattery, 
O dre; ) 


оше gn made to feel ashamed, and to 
On those c ned with the effect he produces 
Years of z эщ him. In bricf, at about four 
definite i n пе child's personality takes on a 

Structure, 


With th i contrasting rather sharply 

sequently earlier undifferentiated picture, Con- 
1 Y. it did sce —À 

children 1 not seem necessary to select 


59:2 younger than four 
in the pr <perin 

е nt experiment, 

The low р e 


lished by D Жан of suitable age has been estab- 
Upper lit. Hi EY process, As for the 
basis of the 5 age, it was established on the 

€ general agreement that the possibility 


Of su 
Ccess " 
Near th Ssful play therapy decreases somewhere 
€ twelfth year of age. 


clus of age for in- 


Intell; 
"lelligence Quotient 


To А 
а “йаз g Considered intellectually satisfactory, 
More than се ord Binet IO score could not vary 
1 Er roa юаш deviation from the nor- 
attempt - Ms requirement was established 
extrem тусте the possible influ- 
bes of ely high and low intelligence on 
statements children might make. 


Belley Scales 


Rati 

“tings c — 

rough К = aggressive behavior were obtained 

Scales axe | „шке of the Beller scales (1). These 
oe ratings of student classroom 

< Th ; A : 

on Such fa € Scales include behavioral items 


as dependency, independence, 
» Submission, 


e and aggression. The 
ееп descri о: SEEFOS . y 
Measure пыш ibed as a reliable and significant 
tating оэ With “little overlapping among the 


B scales use, 


d separatel "n. 62). 
voy, С Sale it E def p. ба) 


Ee ems determining aggression in- 
a depen of verbal and tn items of physi- 
Cited h ssion. The highest positive correlation 
буд, Beller is that between these items. 

» Such as the present one, con- 
types of statements rather than 
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physical indices of aggression should pay atten- 
tion to the verbal habits of the children selected 
for study. More scale items concerned with ver- 
balizations would seem to be a desideratum. 
Consequently, Bellers dominance scale, con- 
sisting entirely of items concerned with verbal 
behavior, was also employed. Beller's dominance 
and aggression scale items have been shown to 
have a statistical relationship. They also have 
a theoretical relationship which may be described 
in terms of the bossing and bullying of aggres- 
sive children. The aggressive child is also the 
verbally dominant child. 

Additional evidence of the relationship be- 
tween dominance and aggression is furnished by 
Griffiths (3). Griffiths’ investigation seems to have 
equated characteristics called dominance by 
Beller with the total concept of aggression. Be- 
cause of the emphasis on speech in the Beller 
dominance scales as well as the close relationship 
between aggression and dominance, both scales 
were employed in the present investigation. 

Both scales have good reliability. Four pairs 
of raters achieved reliability figures ranging from 
78 to .97 for items concerned with verbal ag- 
gression, and .81 to .97 on physical aggression. 
"The scale items measuring dominance yielded r's 
of agreement ranging from .86 to .97. Both 
scales include only behavioral items readily 
apparent from the child's interaction. in the 
classroom. The scale items make no attempt to 
measure the child's attitudes toward his teacher 
or classmates. 

In the present experiment the children's teach- 
ers served as their raters, and the children's 
classroom behavior furnished the source for their 
rated behavior. 

Beller (1) obtained groups of children by 
selecting for his experiment those with high and 
low scores on the scales. A high score was de- 
fined as a mean rating score of 5 or more and 
a low score as a mcan rating of 3.5 or less. The 
present. experiment followed his method with 
the exception of introducing an intermediate 
group, i. those with scores falling. beyond 
3.5 but not reaching a score of 5. The inter- 
mediate group was selected in an effort to give 
continuity to whatever trends might be found. 


Number of Subjects 


Just as the senior writer did not rate the 
subjects of the present experiment, in order to 
avoid gaining any preconceptions about a child's 
aggressiveness, SO he did not examine a child's 
Beller scales until the child's play therapy ses- 
sions were ended. Upon completing his play 
therapy experience the child's Beller scale was 
scored and he was assigned to the aggressive, 
intermediate, or nonaggressive groups. 

There were 20, 22, 24, and 23 children aged 
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4, 6, 9, and 12 years respectively and 26, 27, and 
ME М ә Жыры E 8 
36 children falling into aggressive, intermediate, 
and nonaggressive categories respectively, 


Nondirective Play Therapy Sessions 


Children found to be chronologically and 
intellectually suitable for inclusion in the pres- 
ent investigation were given three one-hour indi- 
vidual nondirective play therapy sessions with 
the same therapist in the same playroom, Three 
sessions were chosen because: | 

1, The child would have had an Opportunity 
to become familiar with the playroom. His time 
would not be spent exploring the room, 

2. The child would have selected his favorite 
play activities, if such a selection were to be 
made, and would be behaving in a characteristic 
manner, 

3. The experimenter would no 
stranger to the child. 


4. The child would have discovered the free- 
dom of the room and would not play in a 
constricted manner if he did not wish to, 

5. The meetings would not have become tire- 
some to the child. 


The therapy records wi 
style, 


longer be a 


ere kept in verbatim 
, records were made during the play 
therapy sessions, both child and therapist. re- 


sponses were recorded, and the material was 
recorded exactly as spoken. This method was 
said to be the most reliable and complete when 
phonographic recording could not be used. It 
was expected that the inclusion of category W, to 
be described later, would make the record proto- 


cols even more complete than they would have 
been otherwise, 


Borke Categories 


The statements made by the children were 
categorized for comparison. The Borke categories 
(8) were employed for this Purpose, These cate- 
gories were originally developed by Helene 
Finke (2) and have twice undergone revision 
and expansion by the senior Writer (7. 8). To 
avoid confusion, the 


categories employed in the 
present study have becn given Helene Finke's 


married name, Borke, The differences betw 
Finke's original, unpublished categories, 
vised Finke categories, and the Bo: 
are treated elsewhere (8). 

One difference not Specifically treated else- 
where is that of recording incomprehensibility 
in some children’s speech. No Previous investiga- 
tion of the process of nondirective play therapy 
dealt with incomprehensible speech as part of 


the category system. Yet it seems unlikely that 
many investigators could have un 


сеп 
the те. 
rke categorics 


s ignor- 


speech should Probably be 


BO 


interpreted either as having recorded omy Revie 
prehensible responses or as having пое: 3i 
incomprehensible remarks with such phrases не 
“words lost here,” Category V of the d 
categories consists of sound effects, xo 
which are not speech but which pertain to s 
child's play. Category W is deliberately set ul АЕ 
make mumbling or talking to self in a vaie \ 
low to be heard a part of the frequency coun 
of the child's category usage. uim 

Twenty-two of thc 644 pages of wer dis 
style records were categorized bv three к ы 
enced play therapists. These pages were sclec уз 
on the h; of a table of random numbe si 
Finke (2), v ing five inexperienced. judges, НЕ 
that the percentages of agreement betw E ced 
original analysis and that of the judges TE n 
from 66 to 77 per cent. In а previous we 
gation (7) the percentages of agreement bet dies 
the criterion judgment and that of two аш ai 
enced judges were 7! and 78 per cent. DE 
centages of agreement between the х nior id t 
judgment and that of two judges pepan of 
play therapy and skilled in the awahana 
protocol were 77 and 81 per cent for the pr 
experiment, 


Ра 


In addition, it was possible to ане рон ше 
similarity of category usage by an over-all 1 a of 
ure of concordance among the three jue E dg: 
the present study, ‘The communality ok tail 
ments for the three observers was Чота 
by the W-statistic (4). The coefficient of con al. 
ance of the rank order of cach category wa "ег 
This figure was found to be significant a ss а 
than the „от level of confidence, Tt таене 
significantly high degree of similarity ation 
rank оге sensitive 
Thus, the Categories have been found aid by 
in the differentiation of statements mace 
children in play therapy. r 

The Borke categories employed in à 
ent study are shown immediately below. 


JANTI- 
HELENE Borkr CATEGORIES FOR QUA? 
* SESS 
FYING THE PLAY "Гиквдрү PROC! 


r of the judges’ 662 categoriz 


the pres 


ings 
A. Curiosity about the situation and as 
Present. in it, (Why did you choose me? / "rew 
else been her Who owns these toys? Who 
that Picture?) 1с 
. Simple description, information, unt is 
ments about play and playroom. (rui 
army. These are prisoners, More marbles. 
тоот” different.) 11 
C. Statements indicating aggression. (А 
erences to fighting, shooting. storms, bur 
drowning, death, hurting, destroying, ete) 
‚БЭ. Story units, (1. Unconnected with я 
Stories obviously farfetched or too cxagge' 


and inconsistent to have occurred. 2- 


om- 
an 
he 


ying 


play: 


ate 
Any 


—#- 
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VERBAL EXPRESSION 
imaginary dialogue or story plot wound around 
the play, such as: He guards the opening. He's 
asleep. He doesn't know they're after him, I'm 
taking them to the army.) 

E. Definite decisions, (Vm going to build a 
bridge. | said Га do it and I did. Just what 1 
Wanted. Did it.) Ё 

V. Inconsistencies, confusion, indecision, and 
ate (My mother has two children, no, one. 
i. brother is half my age and he's much taller. 

"мег birthday was the day before mine last 
aY but mine is before hers this year. Tm not 


Sure wh: Й : i 
€ what I should do. | wonder if this will 
Work.) 


б. EN 
(Can т; 


bloving the limits of the playroom. 

Paint Н аке this home? Can I get wi er? Can I 

I can M l'm going to take this; One second. 
“an Мау longer.) 

Pee cin. to shift responsibility to the 

eno; es (What should I do next? Is this deep 
"Bh? Ts this good? Do vou like this?) 

you. ae of interest in the counselor, (We 

You? ©; Yesterday? W hat Чо vou do? How are 

such rw I trust you? Have you read such and 

а book?) 


th J Attempting to establish a relationship with 
th MM (Guess, Bet vou can't guess. What's 
xoi, DO at that. See. Do you know what m 


you Ms do? Want to see how cars crash? Will 
` K cip me? You do this and VII do that.) 
» Negative 


dumb. Pin ; statements about the self. (Tm 
bh Posi afraid. I never win.) | 

Bood in n Statements about the self. (Vm 

ск, p chool. T can do that. T play marbles 


ru Win it back.) 
+ Negative Statements about the family. 
Nie ы & made or present in the playroom, 

а, activities, elc. (Is there going to 
п sand? т wish this was bigger. I don't 

ler, I wish I had more tovs at home.) 
ve statements about the family, 


i activities, etc, (I like it here. This 
O pretty, We just got a wonderful new 
M home.) 
lamin тай information and stories about the 
big he 00l; pets, teacher, self, ete. (We have 
' a C. T went to the ратк yesterday. I 
You ү, T was waiting for vou. I thought 
MP Qi mother.) 
Pars? Wha & for information. (Do birds have 
a, 2: Dues 's the paint? How docs this work?) 
ring the ons or comments pertaining to time 
De have? 7 aent, (How much longer do 
" lave Det there are fifteen minutes left. 
+ Exel time to play?) 
Dary Gy mations, (Here we go again! Hey! 
melasia Ahhh!) | 
Ye. Fx fiable. (Yes. Mmmmm. OK. Hello. 
© Neuse me, Any answer to a question 


Puppy ; 


Sister, 
ere 
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Gt 


or a pure repetition of counselor's words.) 

T. Insightful statements revealing self-under- 
standing. (When 1 worried it made me steal, I 
wasn't loud but I was mean.) 

U. Ambivalent statements. (Vm scared in here 
but I like to come here. I'd like to paint now 
and blow bubbles too.) 

V. Sound effects, (Vocalizations which are not 
speech, Such noi as clucking, siren, machine 
gun, explosion, airplane, etc.) 

W. Mumbling or talking to self in a voice 
too low to be heard. (Statements which cannot 
be heard and which the child does not direct 
to the therapist.) 


RESULTS 


‘The extensive data of the present in- 
vestigation indicated that a normal dis- 
tribution did not prevail for the cate- 
gories. 

Bartlett tests were applied to 16 of the cate- 
gories. All but one of these tests were significant 
at better than the .o1 level of confidence; the 
remaining test was significant at better than the 
юз level. The data were transformed by both 
the square root and logarithmic methods in 
an effort to change the scale measurement to 
one in which the variance would be more homo- 
geneous, i.c.. less of a departure from normality. 
‘The square root transformation yielded signifi- 
cant Bartlett tests on nine of the categorics. The 
log transformation gave 10 significant Bartlett 
tests, Six of the categories reached significance 
in both cases. 

Evidently a statistical test that did not require 
the assumption of approximate normality was 
needed to test the hypotheses of the present 
experiment. The Kru al-Wallis FJ test was used 
(5). This nonparametric statistic seemed to be 
the most sensitive of those available for applica- 
tion to the present data, 


Results of a one-criterion variance 
analysis for ranks (based on frequency of 
Borke category usage) in relation to ag- 
gressiveness (k = 5) indicated that all of 
the categories save T and U reached sig- 
nificance. A similar analysis for rank in 
category usage in relation to age (k = 4) 
yielded significant 77 tests for all but 
four of the categories. 

In every case of significance those chil- 
dred demarcated as aggressive had a 
higher rank-order (based on frequency of 
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TABLE 4 


MEAN RANK OF PERCENTAGE OF CATEGORY USAGE BY CHILDREN IN Е 
: WITH THE H TEST FIGURES AND THEIR SIGNIFIC 


UR AGE GROUPS, 


| Age 
| “ v | 
Category | H Sig. Leve 
| 
А. Curiosity Antea vi 52 46 36 48 Nanig. 
B. Description... . 56 47 39 30 көр 
С. Aggression. . 49 55 ED 28 è 
D. Story Units.. 50 58 46 28 ЕС 
Е. Decisions. 51 50 48 33 Nonsig. 
F. Doubt.... 35 49 53 41 Nonsiy. 
G. Exploring Limit 63 50 4o 20 
H. Shift Responsibility, 40 39 46 40 Nonsig. 
r Interest... sorena Ge 48 38 35 
2 04 50 39 20 
K. 44 56 17 34 
L. 54 45 49 34 
M. 44 52 H 40 
N. 44 49 47 40 1.76 
р 4c 51 4o 2 CRT d 
Q. 32 38 38 54 7.71 
d бо 54 ds 34 17.40 . M 
P 41 53 47 30 1.12 Nonsig. 
S d 
T. 23 39 40 66 30.25 :01 
Ш 47 46 49 38 4.16 Nonsig. 
W 52 57 43 30 16.00 01 
| 43 55 52 3o 17.23 .o1 


Note.—All mean ranks have been rounder 
gories, S (unclassifiable Statements), 
reaches maximum relative usage by the 
nonaggressive children. Table 4 reveals 
11 significant categories; of these, eight 
of the categories are significant also in 
Table $. Category S, once again, is em- 
ployed relatively more frequently by a 
nonageressive group; this time it is the 
12-year-olds. 

Application of the H test to ranks 
based on relative frequencies уу 
to sharpen its sensitivity to the 
enabling it to pierce the massive verba] 
wall of the aggressive children and the 
6- and 4-year olds. The results obtained 
from such usage will be those 
in detail. 


ould seem 
extent of 


discussed 


Discussion 


The outstanding finding of the present 
study would seem to be that aggression 


d to the nearest whol 


e number, 


and age exert a marked influence on the 
amount and variety of speech produced 
by normal children in the nondirective 
play therapy situation, So pervasive $ 
this influence that future research in 
nondirective play therapy should seck to 
control these factors, Aggression and age 
should also be considered in theoretical 
formulations of nondirective play ther- 
ару. Rather clearly, not all subjects do 
make the same types of verbalizations Mi 
nondirective play therapy, The “regular 
ity" of the process of nondirective play 
therapy is Seriously questioned. 


i vies pi zell 
in aggression, and nonaggressive as W 


as ia ges at different age levels. ie 
n the basis of aggressiveness it was hypothe- 
sized th; - aBgressiveness it wa Mi 


2 : oup. 
dren in the aggressive eo 
aggressive statements (ca a 
n the rules of the playroo 


> 
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(category G) to a greater extent. than would 
children with teacher ratings of intermediate 
aggression or nonaggression. Table indicates 
that both these differences are significant at the 
91 level of confidence in the hypothesized di- 
rection, Aggressive children also voiced. signifi- 
cantly more definite decisions (category E) and 
made more exclamatory statements (category R) 
than did children in lesser aggression categories. 
The first hypothesis received complete support by 
Significant H tests. 

Aggressive children also told significantly more 
Stories (category D) than. did other children. As 
Was hypothesized, the aggressive children evi- 
denced significantly more interest in. the. coun- 
Selor (с: tegory D) and did make significantly 
More attempts to establish a relationship with 
the therapist (category J). 

It is interesting to note the findings in re- 
gard to categories K and L (negative and positive 
Statements about the self, т »ectively). The ag- 
Sressive children made significantly more of both 
kinds of statements, This finding is interesting 
“псе it was hypothesized that it was the nonag- 


Bressive child who should make more K state- 
ments, 


Tt seems possible that aggressiveness, in normal 
children, may be related to willingness for self- 
exploration, or at least willingness to verbalize 
Pinion about the self. Consequently, in normal 
Children aggression, like intelligence (6), mav be 
nU z in ight. The important question of the 
БУ thc о | this relationship cannot be nswered 
binal ie at investigation—first, because only 
ees children were used as subjects, second, 
T) urn (98 few insightful statements. (category 

€ made to be considered. 
sie бон to the prediction that nonaggres- 
зы ыш, would more frequently make un- 

. dole statements about themselves, hypothesis 
Chik Contained the expectation that aggressive 
respon ү would make fewer attempts to shift 
Would liy to the therapist (category H) and 
жш significantly more expres ons of 

Моп and doubt (category The differ- 
nificant. the usage of category H were nonsig- 
slight ш intermediate group, in fact, had а 
erence ‘crease over the aggressive group). Dif- 

: in expressions of indecisions and doubt 

also nonsignificant. 


ТЕ wa i 
TE was further hypothesized 
Slgnated ай 


ences 


that children 
Coiiventionali nonaggressive would make more 
ватана 1100 expressions (category S) than 
Ported € children, This hypothesis was sup- 
ed by the H test, 
sion ildren with ratings of intermediate aggres- 
" Sone TE in the aggressive group made sig- 
id the. more comments about their play than 
„С honaggressive children. As can be seen 
Table 3, this was the only time a signifi- 


Tom 
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cant difference was achieved with the inter- 
mediate aggression group having higher rela- 
tive mean rank usage than the group designated 
as aggressive. 

As was hypothesized, there was a significant 
difference in the employment of statements in- 
dicating aggression (category C) at various age 
levels, with the six-year-old children making 
the most such statements, In turn, as was also 
hypothesized, they were followed by the four- 
year-olds, by the nine-year-olds, and, lastly, by 


the 12-year-olds, who made the fewest such 
statements, 

Twelve-year-old children were further hy- 
pothesized to make fewer definite decisions 


(category E) and to make more verbal attempts 
to shift responsibility to the therapist (category 
Н). Neither of these hypotheses was supported 
by the H test (though the differences were in the 
expected direction). 

Because the aggressiveness of the four-year- 
olds was said to be more diffuse, it м hy- 
pothesized that this age-group would make more 
statements indicating а testing. of playroom 
limits (category С). Table 4 indicates that this 
hypothesis м borne out, With some slight de- 
parture from linearity a marked reduction is 
evident from one age-level to another. The 12- 
year-olds were also hypothesized to confine their 
curiosity to questions (categories A and P), The 
differences among the four age-groups were 
found to be nonsignificant for category А (curi- 
osity about the situation and things present in 
it). The 1g-year-olds, however, did employ this 
category more frequently than did the 6- and 9- 
year-olds. Category P (asking for information) 
\ similarly nonsignificant. Here the 12-year- 
olds achieved a slight lead over all other age- 
groups in category frequency. ‘Twelve-year-old 
children. evidenced significantly less interest in 
the counselor and made fewer attempts to estab- 
lish a relationship with him (categories I and J) 
than did children at other age levels, as was 
hypothesized. A significant difference in state- 
ments about the self was also obtained. As was 
hypothesized, the 12-year-old children made the 
fewest such statements, 

Twelve-year-old children did employ signifi- 
cantly fewer story units (category D) than chil- 
dren at other age levels. The 6-year-old children 
made most use of story units, as hypothesized. 

While 12-year-old children may be more adept 
at conversation than 4- and 6-vear-old children, 
they did not offer more information about their 
family, school, pets, ctc., than younger children. 
The H test figure for the category including such 
statements (O) was insignificant. However, on 
the positive side the 12-year-old children did 
utter significantly more polite but meaningless 
phrases (category S). did make significantly fewer 
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sound effects (V), and did less talking to them- 
selves (W) than did children at younger age levels. 

The significant group-differences in categories 
V and W are not surprising in view of the fact 
that their addition to the revised Finke cate- 
gories was primarily to “serve to emphasize dif- 
ferences in speech habits between young and 
older children” (8). They were originally in- 
cluded on theoretical grounds. They received in 
this experiment their first empirical baptism. 
Interestingly enough the factor of aggressiveness 
did not influence them significantly, 

Although no hypothesis was advanced con- 
cerning category Q (questions or comments per- 
taining to time during the interview) this cate- 
gory is one of the significant ones of Table 4. 
Perhaps category О is significantly less fre- 
quently used by 12-year-olds for the same reason 
that S is employed more frequently, That is to 
say, perhaps the 12-year-old children realize it 
might be construed as impolite to ask, “How 
much longer do we have to stay in the play- 
room?” Usage of category Q may be a matter 
of aggression with the younger children who wish 
to assert themselves and to indicate a desire for 
"Is it time yet?" Or, 

mpler than these theo- 
9- and 12-year-olds may 
atches than the 4- and 


r not have had to ask 
questions pertaining to time, 


In summary, {һе 
in Tables 3 and 4 
1. The speech о 
tained significantly 
play, aggressive statements, stor 
decisions, explorations of imits, c 


aged 4 or 6, and someti 
contained significantly 
ments, story units, explorations of the limit 
dences of interest in the 
establish a relationship wi nselor 
tive statements about the self, positive nega- 
ments about the self, comments Pertaini З= 
time, sound effects, and significantly More о 
ments which could not be heard than ra 
children aged 12 years, id the 
4. The speech of the 
tained significantly fewer 
(Yes, Hello, OK) than di 
olds. 


younger chi] 
unclassifiab], 
d that of th 


dren con- 
€ remarks 
€ 12-year. 


It is interesting to notice that of the 11 sig- 
nificant differences indicated on the basis of 
aggression (Table 3), 8 of them also reached 
significance on the basis of age (Table 4). Once 
again the close correspondence between age and 
aggression is made manifest. 


It would seem to be wise for nondirec- 
tive play therapy to recognize and allow 
for the factors of aggression and age in 
its theoretical schema, if accurate pre- 
dictions are to be made in that system. 
Without considering age and aggression 
the research methodology of nondirec- 
tive play therapy may be seriously im- 
peded. The recognition of the place of 
aggression in a child's normal develop- 
ment, both as a normal innate impulse 
and a normal expression of behavior, 
would be helpful in removing nondirec- 
tive play therapy from the morass of 
procedural sameness that threatens їр ak 
present. The therapist seeks to modify 
the destructive element of aggression, 
while permitting the child to utilize this 
basic impulse in the service of a well- 
directed drive, It would seem that non- 
directive play therapy, to the extent that 
it sets realistic limits in an atmosphere 
of respect and confidence, already ful 
fills this therapeutic requirement. How- 
ever, the theory of nondirective play 
therapy has been unaware of the role О 
the child's aggressive impulses and has 
failed to take into account differences 1? 
age. It is time both these factors Were 
Incorporated into its system. 


SUMMARY 


Nondirective play therapy, a method 
which arose from the postulates of client- 
centered therapy, does not formally co? 
side the influence of aggression ОГ age 
In its method, Studies of the process sio 
Nondirective play therapy, whether [| 
tocol-centered or child-centered, genet?" 
ly reveal the belief that children under8? 


* 
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the same process, no matter what their 
age. The factor of age usually has been 
Ignored or poorly controlled, while the 


factor of aggressiveness has not been con- 
sidered. 

From a theoretical consideration of 
aggression and age certain hypotheses 
were formulated. The hypotheses were 
then tested on a group of 89 children of 
Normal intelligence selected on two 
bases. The first of these bases was that 
of age; there were 20, 22 
dren aged 4, б, 9, and 12 years respective- 
ly. The second basis for selection was 
teachers’ ratings of classroom behavior. 
According to these ratings, 26 children 
Were designated as aggressive, 27 as inter- 
mediate in aggression, and 36 as nonag- 
Bressive, These children were given three 
one-hour individual sessions of nondirec- 
Чуе play therapy with the same thera- 
Pist, in the same playroom. Verbatim 
Style notes were made of their speech and 
vocalizations. These protocols were clas- 
sified according to the Borke categories 
(8). 1t was according to these speech cate- 
Bories that specific predictions were 
Made: 


24, and gg chil- 


gress; SE essive children should make iore а 
Xpress; Statements, threats to playroom К ез, 
попа tons of decisions, and exclamations than 
5 88ressive children, 
*. The speech of aggressive. children should 


Contai 
eniin тоге story units than that of other 
lildren, 


Т Жн aggressive child should make more 
More Fie about himself, evidence 
attempt Crest in the counselor, and make more 
counsis to establish a relationship with the 
"MOT than nonaggressive children. 
quena aggressive children should more fe: 
able ee aggressive children make unfavor- 
indecision wont about themselves, express more 
п and doubt, and make more attempts 
5 responsibility to the counselor. 
епі s BBressive children should have mors 
ssi, nal expression in their speech than ag 
© children, 
jority өр carol children should make the na 
© follo aggressive verbalizations. They wou 
Wed closely by those children 4 years old 


© shit 


5. 
Cony, 
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who, in turn, would be followed by those g years 
old. The 12-year-old children should make the 
least use of speech falling into this category. 

7. Younger children were hypothesized to make 
more of their own decisions about their play- 
room activity and attempt to shift fewer re- 
sponsibilities to the therapist than the 12-year- 
old children, 

8. Four-year-old children should test the 
limits of the playroom more frequently than 
older children. Twelve-year-old children should 
express more curiosity about the playroom and 
ask for more information than younger children. 

9. Younger children should make more at- 
tempts to relate to the counselor, evidence more 
interest in him, and make more favorable state- 
ments about themselves than children 12 years of 
age. 

10. Twelve-year-old children should employ 
fewer story units in their speech than younger 
children. Children 6 years old should use the 
most story units. 

11, The 12-year-old child should utter more 
phrases such as, "Goodbye," and, "Excuse me," 
than younger children. The 12-year-old should 
also speak of his family, school, pets, etc., to a 
greater extent, and do less talking to himself 
and make fewer sound effects, than younger 
children. 


The results of a one-criterion variance 
of analysis of the relation between cate- 
gory usage and aggressiveness, and be- 
tween category usage and age, revealed 
significant differences in connection with 
a large majority of the categories. Ag- 
gression and age apparently made such 
an overwhelming difference in category 
usage that the aggressive children and 
the 4- and 6-year-old children employed 
more of all the speech categories demar- 
cated as significant by the H test. These 
children, evidently, did so much talking 
that the specific hypotheses were ob- 
scured in a verbal fog. To discern mean- 
ingful differences, the H test was em- 
ployed to study differences between 
ranks based on the percentage-frequency 
of use of each category by the three ag- 
gression-groups and the four age-groups, 
respectively. 

From this analysis it was found that 
hypotheses 1, 2, 3, and p received statis- 
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tical substantiation in the three aggres- 
sion-groups. Hypothesis 4 did not. The 
part of the hypothesis concerned with 
negative statements about the sell not 
only failed to be supported but was 
flatly contradicted. The possible impor- 
tant applications of this contradiction 
were discussed. 

From the analysis based on the per- 
centage-frequency of use of each cate- 
gory by the four age-groups, it was found 
that hypotheses 6, 9, and 10 were sup- 
ported, Hypotheses 7, 8, and 11 did not 
receive complete statistical confirmation, 
However, the trends were generally in 
the expected direction, 

Most of the hypothesized differences in 


speech categories were fully or partially 
substantiated. These substantiations sug- 
gest that the theories advanced concern- 
ing aggression and age can be utilized 
to predict the types of statements finding 
predominant expression in nondirective 
play therapy. The process of nondirec- 
tive play therapy, as indicated by chil- 
dren's verbalizations in the playroom, 
does not seem to be the same for all chil- 
dren. Children of different aggression 
and age-groups respond differently, and 
in a predictable manner, Consequently, 
it is suggested that the factors ol aggres- 
sion and age require explicit considera- 
tion in the formal theoretical structure 
of nondirective play therapy. 
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INTRODUCTION 


1" MosT studies of problem solving a 
general description of the problem- 
solving process has been arrived at in 
terms of the solution of the problem. 
There stil] remain several questions that 
Would illuminate the problem-solving 
Process in terms of the problem solver. 
For this investigation, there were se- 
lected two processes, related to "direction 
ri Problem solving” and to “adaptabil- 
com first process, Whole-Part Ap- 
individ З сопсегпей with the way the 
ated ee du pues ТЫ 
Phasize process, Flexibility-Rigidity, em- 
While so the way the person functions 
Solving problems. 


MEANING AND DEFINITION OF 
PROBLEM SOLVING 


I ow 2. $ 
fib М this investigation, the meaning of 
um lem solving is limited to one that 

DU + n 
"A help explain the behavior of the 
о blem solver. Brownell’s (3) definition 

Prob n xd. 
= Ое solving was adopted. Accord 


e; 
s monograph is based on a dissertation 
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ing to this definition, problem solving 
“(a) refers only to perceptual and concep- 
tual tasks, (b) the nature of which the 
subject, by reason of original nature of 
previous learning, or of organization of 
the task, is able to understand, but (c) 
for which at the time he knows no direct 
means of satisfaction. Problem solving 
then becomes the process by which the 
subject extricates himself from his prob- 
lem." In short, problem solving involves 
the goal-directed character of thinking, 
an obstacle to thinking, and a solution. 
But to delimit the definition further, this 
report restricts itself primarily and essen- 
tially to the problem-solving process in 
itself. 


s ro BE INVESTIGATED 


PROCESSE: 
Whole-Part Approach 


The Whole-Part Approach determines 
one characteristic of the method used 
by the subject in problem solving. It has 
been noticed that some subjects ap- 
proach problems as a whole while some 
others deal with parts of the problem. 
Thus, some persons, upon being shown 
a picture, will react to it in its entirety 
by calling it, ер, а harvest scene, thus 
approaching the picture as an integrated 
unit or as à whole; others, seeing the 
same picture, will describe a segment or 
part of the picture by noting, eg. a 
wagon, a man, corn stalks, thus ap- 
proaching it through its parts 
ne Research 
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Importance of the Whole-Part Approach 
The Whole-Part Approach is an n 
portant problem-solving process for vari- 
ous reasons, It gives clues as to the їйїн: 
vidual’s perception of the problem, his 
ability to organize parts into wholes. In 
addition, it seems to be related to per- 
sonality variables and thus could account 
for individual differences in problem- 
solving behavior of people with similar 
intelligence-test scores. 
Summarizing the discussion of earlier 
investigations which used concepts re- 
lated to the Whole-Part Approach, the 
following findings seem to be important 
for the present study: 
a. Several investigators h 


of Whole-Part 
describe 


ave used the concept 
Approach or similar concepts to 
the problem-solving behavior of normal 
and abnormal people. 

б. There seem to be individual differences in 
the degree or extent to which normal individuals 
use the Whole or the Part Approach, 

c. The Whole-Part Approach scems to be a 
process that can be observed in a variety of 
situations, 

d. There is s 
Part Appro 
а 


ome indication that the Whole- 
ach is consistent for the indivi 
ty of situations, 

e. It is a process that has mear 
day life situations, 

f. It is a process that seems to h 
cal meaning and seems to be rel 
ality characteristics, 


dual in 


ning for every- 


ave psychologi- 
ated to person- 


Operational Definition 


of Whole and 
Part Responses 


The review of investigations using the 
Whole-Part concept identified a variety 
of behaviors that have been taken 
dence of Whole-Part Approach in a 
variety of problem-solving situations, For 
convenience, the Appendix shows these 
behaviors classified as to Whole and Part 
Responses. This table will serve as a 
basis for arriving at an operational defi- 
nition of Whole and Part Responses for 
the present investigation. 


as evi- 


These earlier investigations seem to 
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suggest several aspects indicative of he 
Whole-Part Approach, namely, (а) ma- 
nipulation of material, (0) speed of solu- 
tion, (c) amount of material щн 
response or solution, and (d) plan of d 
tack. The Whole-Part Approach as meas- 
ured in this investigation combines ques 
aspects. Considering the main — 
identified above, the characteristics O 
Whole Responses as well as Part Re- 
sponses will now be described. " 

Whole Response. The subject usual у 
does not manipulate material in a per- 
formance test but tries to think of a plan 
or a solution. He also proceeds to solve 
the problem quickly without m 
analysis, His responses include the whole 
arca of the problem, or he may porua 
responses combining a great number 
parts. Another characteristic is that iur 
individual formulates a plan of action 
which he usually verbalizes. nm 

Part Response. An individual м, 
the Part Approach manipulates materia 
readily and often produces responses a 
a random fashion. He usually i 
longer time working out a problem -— 
by step. The subject. responds ому s 
one part of the problem at a Ba 
may break up the problems into P e 
and work out each one separately. ly 
produces responses which combine v 
a small number of given parts. He usua f 
has no particular plan of attack in ы 
and when he verbalizes a plan it is only 
in terms of workin irt of the 


8 out one р: 
problem, 


ids 
M " word 
t distinguishes between the Y 


F le- 
and "Approach" as used in МПО 


This repor 
“Response” 


" r D am 
Part thinking, A Whole Response or à m 
Response is 4 Single observation. Whole i t 
proach or a раг I par 


I t Approach is the pattern art 
ing from a Variety of responses, A What а 
Approach represents the over-all process tha 


de Pi : con 
person exhibits during problem solving. The 


= g stg; are 
clusion 18 not to be drawn that subjed> at 
Classified as Using either the Whole or the 
Approach, 


roach 
Rather, the Wholc-Part Appr? 


R { 
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represents a continuum ranging from prepon- 
derantly Whole Approach through Whole-Part 
Approach to predominantly Part Approach. 


Flexibility-Rigidity Process 
If one observes individuals solving 
problems in the course of their daily 
lives, one is aware of the great differences 
in the way people deal with problems. 
While some people will attack а prob- 
lem in various ways, others are unable 
to shift to another attack even if their 
Present performance does not bring the 
desired results, Some people can think 
By of one possible solution when they 
are confronted with a problem, while 
others are flexible enough to think of a 
mol - possible solutions. Certain per- 
Жатын r characteristics and emotional 
е ipe limit the person's Flexibility, 
ttt enn People with similar intelligence- 
their D may be decidedly different in 
problem-solving behavior. 


pl "tional Definition 

айу 
i the purposes of this investiga- 
Тен Ша identification of Flexibility- 
he a i. 3 problem-solving process. 
servation lasis was, therefore, on the ob- 
е fed; of overt behavior which could 
бамы infer Flexibility or Rigidity. 
© appl чиа (6) view of Rigidity seemed 
ул. mainly to impaired individuals 
ion an extreme degree of persevera- 
k hile Fisher (4) limited the concept 
ative o ity to situations where “alter- 
iii, oo of behavior" are charac- 
Of a successful problem solution. 


of Flexibility- 


n 
т er ee 
ae blem-solving situations both these 
sidered of Flexibility have to be con- 


tinue to "i only must the subject con- 
н Sine another way of approaching 
ful, in lon when his attack is unsuccess- 
sib] at times he must find several pos- 


Solutions to a problem. Therefore, 
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the concept of Flexibility in problem- 
solving situations should include variety 
as well as variability of behavior. “Va- 
riety” would then stand for alternative 
solutions of the problems, while “varia- 
bility" stands for behavior free from per- 
severation or behavior conducive to dif- 
ferent attacks оп the problem. 


It is one of the hypotheses of this investiga- 
tion that the structuring of a problem is an 
important aspect of the problem situation. If 
the problem is relatively unstructured as, eg., 
in the Rorschach Test, variety and variability 
of response are possible and constitute an indi- 
cation of flexibility. On the other hand, in 
several other tests, e.g, the Function Test, as 
mentioned later, the subject is required to arrive 
at alternative solutions and thus must show 
variety as well as variability of response in order 
to solve the problem. 

In order to be able to deal with a number of 
different problems these aspects of flexibility 
have to be considered and must be included in 
an operational definition of Flexibility-Rigidity. 
For the purposes of this investigation it was 
found more advantageous to observe problem- 
solving behavior in terms of Rigidity, or lack 
of Flexibility. The operational definition of 
Rigidity used in this investigation is related to 
that of Werner (14), but has been adapted as 
follows: Rigid behavior does not show variety 
or variability where variety or variability is either 
possible or required. 

It should be pointed out that the cvaluation 
of variety of responses with respect to Flexibil- 
ity-Rigidity has to be relative according to the 
particular. population used in the experiment. 
"Therefore, Rigidity of behavior does not neces- 
sarily mean pathological behavior but rather a 
decided difference of behavior from the rest 
of the subjects, Norms that have been set up 
to distinguish between normal and abnormal 
problem-solving behavior are not valid for this 
type of investigation that deals with the prob- 
lem-solving behavior of normal subjects of 
superior intelligence, 


Signs of Rigidity in the Experimental 
Situation 

Before discussing the lack of Flexibility 
in the individual test situations, it is 
necessary to identify in general the be- 
haviors that may be used to infer rigidity. 
Not all of the behaviors may be observed 
in the test situations. However, this dis- 
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cussion will serve аз a guide to the ob- 
servation of rigid problem-solving be- 
havior during the experiment. 


1. Use of a similar attack on different pd 
lems, as expressed through: (a) a decided pre - 
erence for certain BODE this ке 
overemphasis on either the Whole ог the iom 
Approach; or (b) an adherence to a particu a 
attack on the problem—either (i) attending lo 
the whole or part aspects of the problem by 
dealing first with the whole problem, then wat 
the parts, and last with the minute details, 3i 
(ii) abandoning a progression from the whole 
to the part aspect of the problem and dealing 
with either whole or part of the problem ex- 
clusively. . Ң 
2. Limited production of responses, in а 

problem situation in which the majority of sub- 
jects is able to produce many responses or solu- 
lions, is a sign of Rigidity. The amount of 
production that is labeled as rigid has to be 
determined on the basis of the performance of 
the majority of subjects. 

3. Absence or restriction of manipulation of 
problem material in order to arrive at different 
responses is another evidence of rigid behavior, 
where such manipulation is appropriate or 
where it is generally true of other equal sub- 
jects. 

4. Lack of conceptual approach is t. 
sign of Rigidity, especially in 
more than average intellectu 
sides changing the perceptual situation by 
manipulation of the problem material, the indi. 
vidual can change the Conceptual situation by 
br "ging to the problem other material that is not 
present perceptually or by looking at the prob- 
lem in a different way, He may assume voluntary 
sels toward the problem; he ma et up hypothe- 
ses which can be tested later on, or he may 
analyze the problem, 

5. Perseveration, The 
person can dis 
indicative of his Flexibility in sol 
There are two types of Perseveration that have 
been observed in this investigation, In the first 
type the subject persists in giving the same 
response; he seems to be unable to restructure 
the situation, and he is relatively inactive in his 
problem solving. In the second type the Subject 
may attempt some other responses but al 
comes back to the one he had made Previously, 
Both types of perseverative behavior are indices 
of Rigidity and are treated together in this 
investigation. | 

6. Lack of persistence in the ex, 
the problem. This is considered evi 
inability of the person to restructu 
lem. 


aken as a 
а person with 
al equipment, Be. 


degree to 


Which a 
response once 


made is 
ving problems, 


ways 


‘ploration of 
dence of the 
те the prob. 


GOLDNER 


7. Presence of fluid, unorganized VERUM 
This is characterized by the person's Jempi 
from one aspect of the problem to menm 9 
abandoning a product and starting Again: 1 
the beginning: by verbalizing fluidity, ов. 
“My thoughts are so quick," "I forgot Ў к Т 
wanted to do," "After looking at it a while 
Bet confused," "I am all lost." 


These are the major signs of Rigidity 
that will serve as a basis for the ши 
tion of a person's Flexibility-Rigidity 
Process in the experiment. 


PROCEDURE 
Population 

-— е ted carefully 
The subjects were selected carech | 

5 А : hows f inc 
With certain considerations in m ў ч 
C Про. moo: ан 

namely high verbal ability, age, supe | 
A : NEN AK vas be- 
intelligence, and availability. It was а 

Е А ° eubiects w 
lieved that a selection of subjects hese 
: RN 108 
were homogeneous with regard to al 
variables would provide a critical © A 
: are individu: 
the hypothesis that there аге indiv м 
dem " ing processes. 

differences in problem-solving proc 
É + 2 in this 
High verbal ability was important Дыш. 
investigation in Order to get suficient ver ds lem- 
tion during the Experiments to infer P ег 
solving proces: Thus, it was decided 8 ad а 
а population that was older and that db 
more than average amount of formal cite su- 
In addition, the Population was to be scores 
perior intelligence. Commonly accepted er 
State that People with superior di سی‎ also 
Breater use of the Whole Approach anc 


Кы һап û 
show a greater degree of flexibility than 
Intelligent group, 


ИЖ was finally decided to select sol 
year college students at the Liner” : 
of Chicago, By taking a first-year wer 
group the chances were greater ol aan 
a sufficiently 1 


; sam 
arge homogeneous sal at 
: : jm 
With respect to the other factors Л 


tioned above, 


A population 
equivalent 
Percentiles ¢ 
Сап Council 
ination for 
Se 


rence 
Was selected with intellige 
to the ninetieth to the une s 
n the national norms on the ne 
9n Education Psychological er 


з This test 
College Freshmen (ACE)? This 


de- 
Tos а H К ean 
University of Chicago the des of 4 
15 20 with a standard deviatio! 


* For the 
rived score 


INDIVIDUAL DIFF 
had been administered about six months before 
the experiments were conducted and it was 
assumed that no significant changes would have 
Occurred in the six months! interval! It should 
be remarked. that students with these scores 
represented a group slightly above the average 
of the general college population at the Uni- 
Versity of Chicago. 


Another restriction to secure а more crucial 


ee of the al hypothesis was that the 
es апке (Q) and Linguistic (1) score on the 
= * Vest should not differ significantly. It 


шш that extreme dillerences in Quanti- 
o Linguistic scores may n Some, ПЕ 
or of : 3n an indication of either special abilities 
fore НО maladjustments, and might there- 
ve e another variable into this in- 
live Wn ‘Thus, only students whose Quantita- 
iig ы Linguistic scores did not differ more 
Were sacas or one-half a standard. deviation 
„т Selected, 
ampe де further. the homogeneity of Ше 
minata. Fesults on two additional entrance 
Months s that had been administered sis 
consideret. E the present experiments, were 
ing буту, m hese were the results on the Read- 
kill ad on Гем and on the Writing 
Whose scop It was decided to exclude students 
one ath y differed more than four points от 
Standard deviation from their respective 


total Аск 

Пр ы Scores, This would further eliminate 
a = MEE Special aptitudes, and provide for 
а More 


Crucial test of the hypotheses. 


1 ће Sample that was finally used in 

legna, в was selected {rom the 

the Seine a Газна of Ex AMAR at 

eis of Chicago. Nineteen stu- 

above cae fitted the criteria mentioned 

ерее с selected to participate in the 
ents 12 men and 7 women. 


V : ‘ 

to oluntary participation was believed 
Woy] . А T 
Provide the proper motivation for this 


Xperiment, 
„etter to e 
Viting 

Periment 


The investigator wrote а 
ach of the selected individuals 
them to participate in the ex- 
at a time convenient for them. 


The p 
D [us А 
fron Population selected had total scores ranging 


Om gg : 
cot 10 25, Therefore, the difference in total 
Stand tween 23 and 25 is only one-half а 

ay iati | 
аа deviation, 


Were dese records of the entrance examination 

Available to the investigator through the 
Of the Board of Examinations at the 
y of Chicago, 


Courtesy 
уер 
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This letter was followed up with a tele- 
phone call to assure participation of a 
large number of selected individuals. 


Rationale for Choosing Tests and 
Their Description 

In order to secure evidence for the in- 
ferring of problem-solving processes and 
to test the hypotheses about individual 
variation and individual consistency, the 
tests selected had to meet the following 
criteria, 

1. Each test should involve a task requiring a 
maximum degree of overt action, including 
verbalization. 

з. The tests should vary in degree of struc- 

ture and should represent а variety of content. 
Variety in degree of structure is important in 
order to test the hypothesis that an individual 
is consistent with respect to his problem-solving 
processes in structured апа unstructured: situa- 
tions. With a variety of task-content, greater and 
accurate generalizations can be made with 
rd to the processes observed. 
3. Each test should be unstructured to such 
a degree that individual ways of attacking the 
problem are possible-and thus, when thev exist, 
individual differences can be inferred. 


mor 


These are some of the major criteria 
to be considered in the selection of 
special tests for the observation of prob- 
lem-solving processes. The following six 
tests were selected for the experiment 
because they appear to fulfill the criteria 
cited above. The tests are presented 
roughly in order of structure, 


Rorschach Test 

This test satisfied each of the criteria. 
It represents an unstructured situation in 
which the subject can attack the prob- 
lem in a variety of ways and can give a 
variety of responses. He can react to the 
whole or only to part of the inkblot. The 
test contains perceptual as well as con- 
ceptual situations. The responses to the 
inkblot reveal the subject’s perception of 
the blot. This is aided by the inquiry 
after the initial administration, 
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To what extent does this test represent a 
problem-solving situation? The subject has been 
asked to report everything he sees in the inkblot. 
He has not only to differentiate certain parts of 
the blot but he has to assign meanings to the 
percepts that are related or fairly close to the 
reality of the situation. He has to decide for 
himself how he wants to deal with the problem. 
No hints are given how the responses will be 
scored. It is, therefore, possible for the examiner 
to determine how the individual sees the situa- 
tion, and how he deals with the problem. 

Evidence of Whole-Part Approach. The evi- 
dence of Whole or Part Responses has been 
stated in earlier investigations. In this test the 
amount of material or area of the card included 
in the response is the determining factor: 


1. Whole Response: the subject responds to 
the inkblot as a whole; he takes in the whole 
area of the blot. 

2. Part Response: the subject responds to parts 


of the inkblot separately; he responds to minor 
parts of the inkblot. 


It is the relative amount of Whole and Part 
Responses in all the 10 cards that determines 
the Whole-Part Approach of the subject. 

Scoring procedure of Whole-Part Approach. 
All 19 records were scored according to Beck's 
method (2), and the number of W-, D-, and Dd- 
responses recorded. The number of W., D- 
Dd-responses given by each subject we: 
divided by the total number of responses of the 
subject in order to determine the emphasis with 
which the subject responded to the Rorschach 
cards. Then the W%-scores of the group were 
arranged in a frequency distribution to deter- 
mine the rating as to Whole-Part Approach of 
the subject. After considering Beck's (2) formula- 
tion of "normal" approach (20% W-67%, 1) 
13%Dd), and observing natural breaks in the 
distribution of W%,-scores, ratings as to Whole 
Approach were given as shown in Table 1. B 
assigning these ratings, a person's Whole-P 
Approach could be compared with his Wh 
Part Approach in the other tests, 


, and 
re then 


art 
ole- 


Evidence of Rigidity: 

1. М9, 35% or more; D%, 80% 
Dd%, 25% or more. 

2. Number of cards with systematic se 
or with no succession: nine or more, 

3. Number of cards with onl 

4. No turning of cards when 
are 35 or less, 

5. Number of movement responses; 
less. 

6. Organization score (7): 20 or less, 

7. F%: 55% or more. 


or more; 
quence 


У Опе response, 
total responses 


three or 
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TABLE 1 
ASSIGNMENT OF WHOLE APPROACH RATINGS 
IN THE RORSCHACH TEST 


W% Whole Approach Rating 
45 and over Very High (VH) 
30-44 High (H) 
15-29 Medium (М) 
10-14 Low (L) 

e Very Low (VL) 


8. Number of determinants used: seven ОГ 
less, 

9. A%: 45% or more. 

Scoring procedure of Flexibility-Rigidity. Each 
record was scored according to Beck's (2) method. 
These scores were then arranged in а table 
and the nine categories giving evidence of rigid 
behavior examined. The decision as to rigid 
behavior was partly determined by a review 
of the available literature and also by the dis- 
tribution of scores for this particular group. 
because the standards that ipplied to abnormal 
behavior were not always meaningful when 
applied to this group. | { 

In order to arrive at а total score of rigidity 
for each subject, the number of categories show 
ing rigid behavior were added for cach indivi- 
dual. Thus, subject No. 6 showed rigidity HT 
three out of the nine categories, and therefore 
received a score of 3. Although a weighting 
scheme was contemplated, an adequate basis for 
assigning weights does not seem to exist at ius 
stage of our knowledge. The total scores wei 
then arranged in a frequency distribution anc 


i RE А 4 : ater 
ratings Were assigned as discussed in а Jat 
section, 


Function Test 


Description, A test was needed that 
would represent a verbal conceptual sit 
uation, that would be relatively unstruc 
tured, and that would bring out à per 
son's Whole-Part Approach. The writen 
therefore, developed the “Functio” 
Test” This test consists of four prob: 
lems. The task in this test consists © 
supplying several different uses for ©) 
box, (b) a broom, (c) pliers, and (@ 
Paper. It represents а relatively uns 
tured situation since no hints ave ghe" 
with respect to possible scoring, num 
of responses, or time limits. 
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Administration. The subject is asked: “What 
are the possible different uses of 
(box, broom, pliers, paper)?” The investigator 
ENS ам ıtim record of the subject's remarks 
during the administration of the test. The subject 
аар with each object until he exhausts all 

he possibilities. There is no time limit. At the 

ч cach problem the investigator makes а 
Ў inquiry to discover how the subject went 
about solving the task, 

Evidence of the Whole-Part Approach. The 
(d of the data indicates that an individual 
the UN the object as а whole by leaving 
iio eae intact, or he can break up the object 
бобе М and deal with the parts separately. 
dealin z 2 7 responded to problem “Box by 
(b) Тар it as а whole: (a) to stand on; 
basker àll it full; (c) mail packages (a) waste- 
сой k (c) deliver in boxes; (f) mail box; (g) 

ae (h) cigarette boxes. 
ен а No. Tgave only one Part. Response to 
e add bonfires,” thus breaking up the 
Part RE NIU edge Subject No. 11 gave as а 
Make so ponse to "Box": 7 ake it apart and 
» Soon RB else.” His Part Responses to 
‘Mother were: “(a) take handle and use it as 
Piece of implement—e.g., if you necd another 
Or what Wood; (b) straw to feed a horse or cow, 
respons ever cats in" Thus, in this test, those 
азд аі described the use of the object 
While res с are designated as Whole Responses, 
9f the ae that deal with the use of part 
the digi o that destroy the configuration of 
Y lative, zo called Part Responses. It is the 
in the ton DE of Whole and Part Responses 
the Мер п situations that determines 

Scoring -Part Approach of the subject. 
oach, ra л: of the Whole-Part Ap- 
ated with = response by the subject was evalu- 

Broach respect to Whole Approach or Part 
above, TI in accordance with the criteria stated 
Sponses he number of Whole and Part Ке- 
сас or fi added, and a total score derived for 
Scores s four tasks in this test. Then the total 
sponges “ich represented the number of re- 
Tespect to we computed for cach subject with 
n ord Whole-Part Approach. —— 
€mphasic er to arrive at an evaluation of the 

Ne individu. the Whole or Part Approach for 
Was divi is ual, the number of Whole Responses 
and tee ce by the total number of responses 
Puted س‎ of Whole Responses сош- 
in a fre hese percentages were then arranged 
assigned eny distribution and ratings es 
аз show, о indicate the emphasis on W-Approac 

> ^n in Table 2. 
еу of Rigidity. This test is mainly a 
em the al situation. In order to solve this prob 
attem subject has to restructure the object and 

Pt to see its uses in a variety of situations. 
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TABLE 2 


ASSIGNMENT OF WHOLE APPROACH RATINGS 
IN THE FUNCTION TEST 


W% 


Whole Approach Rating 
05-100 Very High (ҮН) 
02- 95 High (H) 
81— o1 Medium (М) 
74- So Low L) 


Below 74 Very Low (VL) 


A person who is flexible will be able to give 
many uses for the object, uses that are different 
in general idea (c.g., box: container, support, 
ctc). Thus, the number of general ideas offered 
becomes an index of flexibility. Subjects who 
offer only 12 general ideas or less in the whole 
test situation (four problems) are considered 
rigid. 

Scoring procedure of Flexibility-Rigidity. Each 
response by the subject was evaluated with 
respect to the general idea represented. Thus, 
responses for the different possible uses of a box 
such as "to pack things," "for mailing," “аз a 
nest," "as a tool box" were scored as belonging 
to the category "container." Responses such as 
"to sit on it," "as a foot rest," "to stand on" 
were scored under category "support." Among 
other categories were “fuel,” "striking instru- 
ment," "wedge," "float," “writing surface.” 

The number of general-idea responses was 
added for each problem and the results for 
each problem examined in order to evaluate the 
individual's behavior with respect to rigidity. 
The scores for cach problem were arranged in a 
frequency distribution, and ratings of High. 
Medium, or Low Were assigned with respect to 
rigidity. 

In determining these ratings, it appeared ad- 
vantageous to assign the High and the Low rat- 
ings to the upper and lower 25% to 33% of 
the subjects, i.e., five to six persons at each end. 
It was thought important also to consider natural 
breaks in the frequency distribution, as well as 
the median score before ratings were assigned. 
A rating of "High" was assigned to the upper 
25% to 33% of the scores, a rating of "Low" 
to the lower 25% to 83%, and a rating of 
“Medium” to the remaining scores in order to 
arrive at a Rigidity Rating for cach problem. 

In order to arrive at a total Rigidity score 
for the individual on the Function Test, the 
ratings of all four problems were added by 
assigning a value of 5 to a "High," 3 to a 
“Medium,” and 1 to a "Low" Rigidity Rating. 
This method of totaling the ratings reflects the 
relative standing of the individual in the particu- 
lar test situation. 
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The total Rigidity scores for the Function 
Test were then arranged in a frequency dis- 
tribution in order to obtain a final Rigidity 
Rating for each individual on the Functior: Test. 
Again the upper 25% to 33% of the scores 
received a rating of “High,” the lower 25% to 
33% a rating of "Low," and the remaining scores 
a rating of "Medium." 


Using the same method of scoring on 
the other tests, it was then possible to 
compare an individual's final Rigidity 
Rating on one test with his final Rigidity 
Ratings on the other tests. 


Anagram I Test 


Description. One way to observe proc- 
ess would be to put the subject in a sit- 
uation where he would have an oppor- 
tunity to manipulate materials. Sargent 
(13) had used anagrams. However, his 
subjects solved the problems without 
overt manipulation of letters, Therefore, 
the writer developed the “Anagram Test” 
consisting of two parts. The test consists 
of 14 wooden letters (CD EEINOP 
RSS T U V), and the subject is asked 
to make words out of these letters, "This 
was a relatively unstructured situation in 
which the subject could do what he 
thought was demanded by the situation. 


Administration. The subject is told: “See what 
words you can make out of these letters!" There 
is no time limit but the test is Stopped after 
about 10 minutes when most of the subjects 
have exhausted the possibilities, Any questions 
with regard to structure are answered with 
“This is up to you.” The investigator wrote 
down the words constructed. The observations 
included the amount of manipulation and place- 
ment of letters and any verbalizations, 

Evidence of Whole-Part Approach. 
using a Whole Approach in this situation will 
attempt to combine a great many letters in one 
word; he will produce big words, A person using 
Part Approach will be satisfied to produce 
smaller words, Thus, with respect to amount of 
material included in the response, the number 
of letters in a word is taken as a Criterion of 
Whole or Part Response. In this experiment a 
word of five letters or more was Considered а 
Whole Response. The division of five letters or 
more was arrived at arbitrarily after а Study of 


A subject 
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the test results revealed that the median was 
four letters. The words consisting of five letters 
and more represented 164; of the total responses 
given by all the subjects. Thus, at fivc letters 
there seemed to be a natural break in the dis- 
tribution, distinguishing large from small words. 
A word of four letters or less was taken as а 
Part Responsc. 

The Anagram I Test was selected as a less 
structured situation, Since the production in the 
first half of the test scemed to be less limited 
than that in the second half, the first half of 
the subjects responses was used to evaluate the 
person's Whole-Part Approach. The data showed 
a slight shift toward larger words in the second 
half of the test (see Table 3). m 

In order then to determine the persons 
Whole-Part Approach in a less structured situa- 
tion, the ative amount of Whole and Ёш 
Responses in the first half of the individual's 
production has been used. An individual who 
starts out. producing small words in the fist 
half of the test may be forced to produce large! 
words in the second half because the production 
of small words becomes less possible. |, 

Scoring procedure of Whole-Part Approach 
In accordance with the criteria mentioned Uo 
all the words containing five or more IO 
were scored as a Whole Response. А score eid 
Whole Responses was computed for the e 
half of cach individual's responses, dividing e: 
number of words with five or more letters. nd 
curring in the first half of the test, by 8 
number of responses in the first half, аз show п 
in Table 3. These scores were arranged in 1 
frequency distribution and ratings we айн. 
with respect to Whole Approach, (Some ИШ 
viduals may remain at the same level as n rai 
first part of the test, ie., they keep on prods 
ing small words, Others produce small words Si 
the second half after having produced bigs 
ones in the first half.) 


TABLE 3 


PERCENTAGE 
or 11 


= ma 
Partof Number of Letters in Wore а 
Test 6 7 

s Ў : Я 2 & Fe 


First Half | 
(Responses! 


67) 


Second Half 
esponses 
=467) 


=— 


| 4% 32% до% 


ر ی m‏ 


15% 70 2 
_ ® qwe 
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Evidence of Rigidity. (a) Those persons who 
produce more than 50°% of two- and three-letter 
words in the first half of the test are considered 
as showing a certain amount of constriction in 
the spontancous reactions to the problem. (b) 
Those subjects who keep on producing 50% OF 
More small words in the second half аге con- 
sidered as showing an even greater degree of 
constriction, (c) Those subjects who show a 
decided increase in production of small words 
In the second half of the test (over. 10*,) may 
have become more constricted when the situation 
became more structured; ог they have 
become less motivated. i 
Rete Problem in this test is to make words. 

E is said about the quantity, quality, or 

x e features. of the words. Nothing is 
should ga: the manner in which the subject 
May co proceed. in constructing the words. He 
the Rates to use the same letters nd change 
If his 2 heb niger ed (e.g. pit. pet. put. pot). 
type e s uction of words is restricted to this 
small on RU he usually produces only 
of east ts This behavior | taken as a sign 
centage Shane in this investigation. If the per- 
ing Rait, mons constructed by this perseverat- 
evidence BM. E and above, it is considered 
of ri 

test oing procedure of Flevibility-Rigidily. The 
With зн for th individual in accordance 
three-letter ne stated above. The two- and 
added up mes produced by the subject WEE 
Well as pi the first half of his production as 
small OE, the second half. The number of 
by the ae in the first half was then divided 
first half E number of words produced in the 
ame pro X І the percentage. computed. The 
Produced qune was used with small words 

Since the the second half. MM 
Produceq i number of large and small — 
alf Was Р the first half and in the second 
loss in us same, the percentage of gain or 
Puted n © Production of small words was com 
Words "m. comparing the percentage of small 

duced in the first half with the per 


may 


mer 
O 


CCntage 

hair Бе of small words produced in the second 
Each 
"Ch recor А 

method record was scored with respect tO the 

duceg With which the words had been pro 
1. Sup; 

. ` Subjec — - 

Pin-spin! t builds up words: c.g. pot—post; 


. 2. Sub; | 
tise oject breaks down words: ege TINSC™ 
% Pride—ride 
3. Sub; | 
the Acie changes vowel in the middle of 

4. Sub. eg., strive—strove; tone—tune. 
Changes ject holds on to same letters 
dip, tri only the first letter: eg, sip—nip—UP— 
apy 'eS—dries—cries, 


еп 
Umber of words produced by cach of 


and 
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these four methods was then added and а score 
computed. In order to arrive at ап evaluation 
of rigidity this score for cach person was divided 
by the total number of words produced by the 
subject and the percentage of words produced by 
these four methods computed. The number of 
repetition of words was also noted for each per 
son. 

In order to arrive at an estimate of a person's 
rigidity in the Anagram 1 Test, the total scores 
of each category were arranged in a frequency 
distribution and rating; igned, as discuss 
in the section above. Similarly, total Rigidity 
scores were computed and then arranged in E 
frequency distribution and final ratings assigned 
in order to make it possible to compare а per- 
son's behavior in the Anagram 1 Test with his 
behavior in the other tc 


Anagram II Test 

Description. This situation represents 
а more structured one than the Anagram 
I Test. The subject is asked to construct 
one word out of all the 14 letters given 
(CDE EINOPRSSTU V). These 
letters form the word “Productiveness.” 
This word had been chosen because it 
was suitable for a variety of approaches 
and seemed to have the correct level of 
difficulty as determined by a few prelimi- 
nary administrations. (Only one subject 
solved the problem during the actual ex- 
periment.) It also lent itself to the con- 
struction of several large words, an im- 
portant aspect in the observation of the 
Whole-Part Approach. 


Administration. The subject is told: “Try to 
make one word out of these letters!” There was 
no time limit but the test usually lasted 10 
minutes. The subject was observed while he was 
trying to construct one word out of all the 
letters. During this time his moving of the 
wooden letters and his verbalizations were re- 
corded. These records were then used to deter- 
mine what the subject attended to and how he 
went about solving the problem. 

Evidence of Whole-Part Approach. This 
represents a more structured situation, as the 
individual is asked to combine all the letters in 
one word. Besides considering the amount of 
material included in the response as in the 
Anagram I Test, the manipulation of the letters 
is taken as evidence of Whole or Part Responses. 
Lack of manipulation of the wooden letters and 
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“thinking of words” is taken as evidence of a 
Whole Response. On the other hand, ready 
manipulation of letters as well as construction 
of words in a random manner is taken as evi- 
dence of a Part Response. Furthermore, with 
respect to the amount of material included in 
the response, the production of prefixes and 
suffixes or working out a small part of the word 
separately is taken as evidence of a Part Re- 
sponse. Subject No. 10 used first a Part Approach 
by arranging letters as to vowels and consonants, 
and then by putting the two S's together. He 
used a Whole Approach by thinking of words 
like “vicissitude,” "visualization." He verbalized 
this Whole Approach by saying that he was 
thinking of several words having sufficient length. 
After seven minutes of thinking of words he 
shifted to a Part Approach: “Maybe a trial-and- 
error process is better-and E in the back is 
probably . . .—assuming it began with а conso- 
nant an $ as initial, for convenience let E follow 
it; let another consonant follow—N-—; I have to 
put more than one consonant together—we'll run 
out of vowels!—'sent'—-that doesn't recall a word." 

The Whole-Part Approach of an individual 
was determined by the relative amount of 
Whole and Part Responses observed and re- 
corded during the performance, 

Scoring procedure of Whole-Part Approach. 
Each individual record was examined in accord- 
ance with the criteria set up above, Thus, every 
performance was evaluated as to evidence of 
Whole or Part Approach. Whenever there was 
evidence of the Part Approach a "D" was re- 
corded. A “W” was recorded when there was 
evidence of the Whole Approach. If there was 
only slight evidence of Part or Whole Approack 
a (D) or (W), respectively, was noted. Then an 
over-all judgment was made as to Whole-Part 
Approach. These ranged from "pure Part Ap- 
proach—D” through "mostly Part Approach 
D(W),” “combination Whole-Part Approach— 
DW,” to “mostly Whole Approach—W(D),” A 
final rating as to Whole Approach was assii 

кыин М as assigned 
to each individual's performance: a rating of 
W(D) and a rating of DW were considered 
^High" in Whole Approach, a rating of D(W 
"Medium" in Whole Approach, and a ratin: н, 
D, “Low” іп Whole Арргоасһ, Ee 

Evidence of Rigidity. (a) Produces only 
fixes. (b) Produces mainly words. 7 pre: 
H y - (c) No produc- 
tion of words or prefixes. (d) No Succession of 
sequence. (c) No manipulation of letters, (f 
Number of words produced (4 or less). (g) Do ) 
not mix up letters in pile. (h) Does not ask iim. 

self questions. (i) Does not set up hypothe ý 
(j) Does not think of words. (k) Makes eee 
ET (1) Thinks of earlier large word, (т) Gives 
Scoring procedure of Flexibility-Rigidity Each 


record was examined as to the presence or ab- 
sence of behaviors that could be taken as cvi- 
dence of rigidity as stated above. In order to 
arrive at a total score of rigidity for cach sub- 
ject, the number of categories showing rigid be- 
havior were added for cach individual. Thus. 
subject No. 1 produced only prefixes, showed no 
succession of sequence, did not manipulate let- 
ters, did not mix up letters in pile, and did not 
set up any hypotheses. His total score was 5. The 
total scores were then arranged in a frequency 
distribution and final ratings were assigned as 
discussed in an earlier section. 


Stencil Design Test II 

Description. This test (sec reference 1) 
was selected because it involves overt 
action in a relatively highly structured 
perceptual situation. It consists of 20 de- 
sign cards in black and white, each three 
inches Square, and 24 colored stenciled 
cards, also three inches square, "Twelve 
of these colored cards are yellow, and 12 
are blue. There are 11 stencils and 1 solid 
card of each color; they are numbered 
in the order in which they are to be ar- 
ranged on the table when presented to 
the subject. All the colored cards are 1€ 
versible so that no time is lost if a card 
is laid face down. Both the design cards 
and stencils are symmetrical to avoid dif- 
ficulty with right-left or up-down rever- 
sals. The object is to superimpose blue 
cards and yellow cards upon each other 
to form a design similar to the опе 9^ 
the black and white design card. 

For this study only five designs were 
selected for the experiment prope” 
namely, numbers g, 13, 10, 18, 19 (2), 
These designs were selected to provide 
the Proper difficulty and different pov 
sible problem-solving approaches for ^ 
Superior group of students. Problems = 
and 19 can be solved by the Whole АР" 
proach whereby the subject has to analy” 
ше design, i.e., determine which ste? : 
are involved, These two problems canno 
be solved easily by trial-and-error OF 
sheer mechanical approach. In contrast 


| 
| 
| 
| 
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problems g, 13, and 18 could be solved 


mechanically, by placing stencils of al- 
ternate colors on top of each other. 


Administration. In the Stencil Design Test 
the subject is shown a sample card and given the 
following directions: “Make a design with these 
stencils so that it will look like the drawing on 
If the subject does not understand 
the task, the examiner shows him that the pat- 
tern could be copied by placing the blue stencil 
on the yellow solid card, The stencils аге те- 
turned to their places, To make certain. that 
the subject understands the directions the exam- 
iner asks him to copy design number 3 as a 
Practice exercise (1). If the subject does not yet 
comprehend the directions he is given design 
humber 5 for further practice. After the subject 
understands the directions, he is given designs 
number g, 13, 10, 18, 19 in that order. The 
order of presentation is held constant for each 
Subject in order to control any possible influ- 
enee of one design on the following onc. Be- 
Sides that, this specific order calls for changes in 
attack so that a mechanical approach to the 
problems has only a limited chance for successful 
Solutions. The subject is told that there is no 
time limit on the task. Furthermore, he is en- 
couraged to verbalize what he is thinking, how 
the problem looks to him and what plans he 
1а5 to attack the problem. If a subject feels 
hampered by talking out loud, he is not prodded. 
oy question. by the subject as to procedure, 
Cle, is answered with, “That is up to you." 
ш кепе of stencils and verbalizations by 
nes n ject during. the performance were re- 
à rded by the investigator, А special notation was 
ссора һу the investigator to record the 
57 E of stencils, This enabled the investiga- 
MONET evaluate the responses after the test was 
еп с апа did not call for immediate judg- 
efer 9 rating of the performance. As mentioned 

М ©, verbalizations were recorded аз меге 
ed glances at the designs or at the stencils; 
тё. нме observations as, Cgo rapid move- 
in ra tieng: were noted in the record 
iret т per Sequence, In addition, the time of 
асот es ic, picking up of a stencil, was 

ed as was the total time taken for each 
Problem, 
a nee of Whole-Part Approach. This is one 
o хы structured situations, and P waray 
are Roce was taken as evidence of Whole anc 
€sponses: 
u mo ulation of material: Lack of ша 

hole 4 stencils was taken as evidents, of a 
E esponse while frequent manipulation 
d.d en as а Part Response. i 
* Speed of solution: Putting stencils together 


Wa 


without much checking back to the design and 
producing the solution rapidly was taken as a 
Whole Response, while arriving at a solution 
alter a relatively longer time was taken as evi- 
dence of a Part Response (13). 

з. Amount of material included in response: 
The following behaviors were considered as evi- 
dence of a Whole Response; the individual states 
relationship of seve stencils; he states in in- 
quiry that he looked at the problems as a whole. 
Evidence of a Part Response: during the in- 
quiry, the individual states that he knew one 
stencil, color, form; he states that he was look- 
ing for a circle, etc.; he notices minor projections, 
points, etc; he places stencils one at a time; he 
divides problems into separate parts; he works 
out parts separately and then combines them. 

4. Plan of attack: As a Whole Response, the 
individual spends a major part of the time on 
the analysis of the design; he analyzes the prob- 
lem in terms of stencils nceded; he verbalizes a 
complete plan of attack. As a Part Response, the 
individual analyzes the problem for each stencil 
separately. 

The relative amount of Whole and Part Re- 
sponses determines the Whole-Part Approach of 
the individual. 

Scoring procedure of Whole-Part Approach. 
The performance of each subject was rated on 
cach of the five tasks with respect to Whole- 
Part Approach in accordance with the criteria 
set up above, These ratings were added in order 
to arrive at a total score for each individual. A 
frequency distribution was made of the total 
scores, and each subject was given a final rating 
with respect to Whole Approach on the basis of 
his position in the distribution. 

Evidence of Rigidity. In order to evaluate 
rigidity of an individual in this test situation 
several behaviors were considered. 

1. The subject makes the same errors and 
shows perseveration in this way. 

2. The subject holds on to the same wrong 
stencils for more than one response. He does 
not recognize his mistake and does not discard 
the wrong stencil. 

3. The subject misses cues. He seems to be 
unable to look at the problem differently. 

4. The subject uses the same attack on the 
problem. 

Scoring procedure of Flexibility-Rigidity. Each 
of the five problems was examined and scored 
separately as to (a) number of same errors, (b) 
number of wrong stencils (form) held, (c) num- 
ber of missed cues, and (d) number of times 
same attack was made on the problem. These 
scores were then combined for each problem into 
a total score by adding up the number of oc- 
currences of these behaviors. The total scores 
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were then arranged in a frequency distribution 
and ratings as to rigidity assigned. It should be 
noted that problems that were not completed 
by the subject received a rating of “High” rigid- 
ity. 

In order to arrive at a total estimate of thc 
individual's behavior with respect to rigidity in 
the Stencil Design Test, the total scores of each 
of the five problems were arranged in one table. 
By assigning a score of 5 to "High" ratings, 3 to 
"Medium" ratings, and 1 to "Low" ratings, a 
total Rigidity score for the Stencil Design Test 
was arrived at. Then the total Rigidity scores 
were arranged in a frequency distribution and 
final ratings assigned in order to facilitate a 
comparison between the different tests. 


Block Design Test 


Description. 'The Block Design Test 
(8) is another example of a highly struc- 
tured test. The four designs used in this 
investigation were presented in red and 
white, each three and one-quarter inches 
square, mounted on a white five-by-eight 
inch card. Sixteen identical blocks, each 
а oneinch cube having a side colored 
with red, white, blue, yellow, blue-yel- 
low, red-white, respectively, were used 
to reproduce the designs. The designs 
are drawn in such a way that a block is 
represented three-sixteenths of an inch 
smaller than in reality. This size was 
chosen because it allowed the design to 
be large enough to be adequately ex- 
amined, and it made the task more diffi- 
cult than if the design had been repro- 
duced in its true size, 

The changes incorporated by the writer in 
the Block Design Test cause it to differ from 
the one used by Kohs (8). One change consisted 


in leaving out the boundary lines which (in the 


Kohs set) gave the designs a square outline 
It was now more difficult to find a : 


beginning 
and to analyze the design. All the designs were 
drawn the same size, in contrast to the Kohs 


test. This leaves it up to the subject to ¢ 
what size he should reproduce. In ad 
order to increase the number of po 
proaches, the subject was always given all of the 
16 blocks, and it was up to him to decide 5s 
use all or only some of the blocks, 

The designs used in this investigation varied 
as to complexity. Problems 1 and 3 represented 


letermine 
dition, in 
ssible ap- 
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designs with clearly defined outlines, while Prob- 
lems 2 and 4 were more complex. Problem 2 
was the only one that had a square outline In 
the other designs the figure was not squi 

Problems 1 and 3 were also chosen because 
they lent themselves to a construction with four 
blocks instead of sixteen blocks. 

With respect to Whole-Part Approach it was 
thought that Problem g would be mainly ас 
tacked by the Part Approach, while Problem 4 
would lend itself more (о а Whole Approach. 
In Problem $ the figures stand out separately, 
while Problem 4 is too much of a whole to 
break it up into any figures. 

Administration. In the experiment, the sub- 
ject was given all the 16 blocks in random ar- 
rangement and also the Design 1, He was told, 
"Make a design with these blocks so that it 
Will look like the drawing on this card." ‘There 
was no time limit, The subject was gain en- 
couraged to think out loud while he was work- 
ing on the problem. 

Movements of blocks and verbalizations by the 
subject were recorded. In order to record the 
movements of blocks, the special notation pro- 
posed by Rubinstein (12) was used. At times, Ít 
became necessary to draw the designs as con 
structed by the subject when his production 
became very different from the expected onc. 
Furthermore, pauses and qualitative observa 
tions were recorded in addition to verbalizations 
and moves. Also, the time of first response and 


total time for each problem were made part of 
the record, 


smooth, If he employs the Part Approach, He 


| m 3 1 е 
may break up the design into parts which 
completes separately; or he may complete Көр 

у after 


problem step by step, checking carefully а. t 
cach step. Usually, in this approach, the subjcc 
follows the pattern or design of the problem. А 

The relative amount of Whole or Part Re 
Sponses determines the subject's Whole-Part Ар" 
proach, 


Scoring procedure of Whole-Part Approach: 


ч in the Stencil Test, the performance of each 
subject was rated on each of the four tasks Will 
respect to Whole-Part Approach in accordance 
with the criteria set up above. These ratings 
Were added to arrive at a total score for eac? 
individua], A final rating for Whole Approach 
was assigned to cach subject on the basis of m 
position in frequency distribution of total score 
_ Evidence of Rigidity. In order to evalua! 
rigidity of an individual in this test situ? 


tion 
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several scores were considered: 

1. The subject has difficulty in size perception 
and docs not adjust to the given design. 

2. The subject docs not change his attack on 
the problems after failure, 

3. The subject abandons 
though it was correct. 

A. The subject. gives up trying to solve the 
problem. 

5. The subject jumps from one aspect to 
another, 


part-product al- 


6. The subject verbalizes fluidity or lack of 
organization, 

7. The number of errors is high. 

8. The subject does not correct errors, 

9. The subject repeats errors, 

10. The subject holds on to wrong blocks. 

s Scoring procedure of Flexibility-Rigidity. 
"ach of the four problems was examined and 
scored separately as to the presence or absence 
of behaviors that could be taken as evidence of 
Mgidity as stated above. These signs were then 
added for cach subject in cach problem. 
In order to arrive at a total estimate of a 
ло behavior with respect to rigidity, the 
added Gk the individual in cach. problem en 
oT and total score computed. This procedure 
ба ised because the range of behavior was not 
Brent enough to make a separate rating for each 
ptem meaningful. ге total scores were then 

“nged in a frequency distribution and final 


TAME ace? : i [ 
йок Т assigned as discussed in an earlier sec- 


pe 


Experimental Procedure 


to кы battery of tests was administered 
sions и? student individually in tw ses- 
sion Њ и ао hours each, In the first ses- 
and ү tes Design ‘Test, Anagram I 
т ; Fests, Block Design Test, and the 
а Test were presented in that 
believed liig sequence of problems was 
ivation i avoid boredom and poor mo- 
tual to mi changing tasks from percep- 
held co verbal ones. This sequence was 
ng nstant for all ol the subjects. 

est ES second session, the Rorschach 
ier er and the inquiry 
he Need The purpose of presenung 
Vas to rschach Test as the last problem 
ic wey certain that the investigator 
i lave any bias in the observa- 


lio 

n : ‘ 

9f problem-solving behavior because 
5 


of information received through the 
Rorschach Test. Another advantage of 
presenting the. Rorschach Test last was 
that the subjects did not suspect that any 
personality variables were being ob- 
served. The Rorschach Test could have 
been detrimental to the experiment be- 
cause it was known by most of the sub- 
jects as а clinical instrument for the 
evaluation of personal maladjustments 
and would have aroused anxieties in 
some of the subjects. 


The recording of responses has been discussed 
in a previous section, It may be worthwhile to 
point out again that initial response time and 
total time for the task were recorded, with the 
examiner using a stop watch. 

The examiner sat at the right side of a 
table observing the movements of the subjects. 
He recorded moves or verbalizations immediately 
on sheets of paper. The investigator had de- 
veloped a shorthand system for the recording of 
moves in the performance tests, The subject was 
asked to verbalize thoughts, but was not urged 
further if it disturbed his performance. The 
room was a cubicle in the Psychological Labora- 
tory of the. Education. Department at the Uni- 
versity of Chicago. It was possible to work with- 
out any interruption. 


RESULTS AND DISCUSSION 
Whole-Part Approach 
Individual Differences 
The group showed variability with 
respect to emphasis on Whole-Part Ap- 
proach in the Rorschach Test, the Func- 
tion Test, the Anagram I Test, the Ana- 
gram II Test, and the Grace Arthur 
Stencil Design II Test. The evidence as 
to individual differences was meager in 
the Block Design Test. For detailed data, 
see reference 5. 
Consistency of Whole-Part Approach 
In order to arrive at a measure of con- 
sistency of the Whole-Part Approach, 
correlations between the various tests 
were obtained as shown in Table 4. Ac- 
cording to Lindquist the value of r re- 
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TABLE 4 
INTERCORRELATIONS ОЕ Six TESTS WITH RESPECT To WHOLE-PART A PPROACH Е 
г Бане eee | "Total Score 
i i ck Minus 
Test Function | Anagram I Anagram 11 Stencil | Block рей шы Tur ME 
5 | -67 
h „25 -40 +40 -58 +40 A 
ee x .00 —.02 —.02 .10 || e 
Anagram I +25 53 .36 T 
Anagram II .29 = °з ai 
Stencil .62 о 
Block | 


quired for significance for 19 subjects at 
the 5% level of confidence is .456, and at 
the 1% level is ‘575 (9). The correlations 
between the Rorschach Test Score 
the total scores of the other five t 
and between the Grace Arthur St 
Design II Test and total 
а coefficient of .6 
These correlations 
195 level. The correlations between the 
Anagram I Test 
between the Block 


and 
ests, 
encil 


tests in the 


ency of Whole-Part Ap 
dividual, as well as sig 
among individuals, 


proach for the in. 
nificant differences 


INTERCORRELATIONS 
ESPECT Ti 


Flexibility-Rigidity Process 
Individual Differences 


The group showed variability with 
respect to Flexibility-Rigidity in the 
Rorschach Test, Function Test, pet 
Bram I Test, Anagram II Test, ш 
Arthur Stencil Design II Test, anc 
Block Design Test (5). 
Consistency of Flexibility-Rigidity Proc 
ess 


In order to investigate whether icon 
bility-Rigidity is a consistent process, CO E 
relations between final Rigidity ratings 
on the various tests were calculated (see 
Table 
tests with the total score yield resu n 
that are significant on the 5% level 0 
better for the Rorschach Test (r — ps 
the Block Design Test (r = .54), and t 


Its 


OF TESTS FoR 19 SUBJECTS wir 
о FLEXIBILITY- ITY 


Unstructured Tests 


à ES Score 
Tests Structured Tests Total Sc eom 
| R A Minus Par 
: F B St АП ular Tes 
schach ; s 
даам ij ae $5 .26 a7 = -54 
cti ы о ш. is : 
Function 2 = de E: " 
Block Design А Шана ЖИ Е Е 
Arthur Stencil Design 83 .34 di 
Anagram II .50 k^ 


. le 
5). The correlations of the sing | 


, 
4 


| 
| 


| 


| 
| 


L 
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Grace Arthur Stencil Design II Test 
(r 58). These results indicate a fair 
degree of consistency of the Flexibility- 
Rigidity Process. | 

Although these tests seem to correlate 
to some extent with the total score, it is 
Interesting to note that there is no sig- 
nificant correlation between an unstruc- 


tured test on the one hand and a struc- 
tured test on the other. The degree to 
which a problem is structured is evi- 
dently of importance. 

The correlation between Flexibility- 
Rigidity in the Anagram I Test and Ana- 
Bram II Test is ло. If the content of the 
test were influential in determining 
Flexibility-Rigidity scores, these two 
tests should correlate significantly. There 
1s thus an indication that content enters 
Only to a small degree into the results 
on Flexibility-Rigidity Process. 

Thi 


tior 5 conclusion is reinforced by the examina- 


fiin s intercorrelations of the Stencil De- 
Bràm "n 5 the Block Design l'est, and the Ana- 
Block p Test; The Stencil Design Test and the 
tion is resign Test correlate 83. This correla- 
by the Ee high апа may be in part explained 
ара НУ of the tests, Both tests include 
relations of materials and analysis of spatial 
the ma; nps, If the content of the test were 
with ee factor, the tests should not correlate 
esign Anagram IL Test. However, the Stencil 

50) Whi est and the Anagram II Test correlate 
confide i is significant on the 5% level of 
tpm dC Fhe correlation between. the Block 

34 est and the Anagram II Test is only 
К nd, therefore, statistically not significant. 

sta Соттоп factor among the three tests is the 
betwen.” of the situations. Thus, the correlation 
Bram T the Stencil Design Test and the Ana- 
Test can be explained not on the basis 


с А 
ontent but on the basis of structure of the 
lation, 


Situ 


p Function Test shows an unusual 
n in that it correlates with some 
n unstructured and some of the 
Plain 2. tests. This can in part be ex- 
Bye и by the differences in structured- 

“ the four problems comprising the 
(ер. “paper” constitutes an object 


р 


with a relatively small degree of struc- 
ture, while "pliers" represents an object 
with a relatively high degree of struc- 
ture). 

These correlations imply that individ- 
uals who are rigid in the Rorschach 
Test may also behave rigidly in other un- 
structured situations. On the other hand, 
subjects who are rigid in the Stencil De- 
sign "Test may behave rigidly in other 
structured problem situations. 


Relationship of Whole-Part Approach to 
Flexibility-Rigidity Process 

The investigation of Whole-Part Ap- 
proach and Flexibility-Rigidity Process 
raises the question whether these proc- 
esses are independent of each other. 
Therefore, product-moment correlations 
were computed; the results are presented 
in Table 6. 

In order to interpret the intercorrela- 
tions it should be remembered that the 
final ratings for the Whole-Part Ap- 
proach signified the degree of Whole 
Approach present, while the final ratings 
of the Flexibility-Rigidity Process repre- 
sented the degree of Rigidity a person 
exhibited in the problem situation. 

When the correlations between the 
Whole-Part Scores and the Flexibility- 
Rigidity Scores for each test were com- 
puted, neither the Rorschach Test nor 
the Anagram I Test showed a relation- 
ship between the Whole-Part Approach 
and Flexibility-Rigidity Process. This in- 
dicates that the measures used in these 
two tests are independent of each other. 

The Anagram II Test, the Stencil De- 
sign Test, and the Block Design Test 
showed a statistically significant relation- 
ship between scores for the Whole-Part 
Approach and Flexibility-Rigidity Proc- 
ess (correlations of —.57, —.54, and —.66, 
respectively). The negative correlations 
indicate that a person who rates high 
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TABLE 6 
VEEN x ‘GS OF W E-PART APPROACH 
STE ELATIONS BETW FINAL RATINGS or Wior 
ни AND FLExIBILITY-RIGIDITY PROCESS 
Flexibility-Rigidity Process 
|o c (= | Bineten | Argan [Апаат | Serai | mu Total 
м H Anagram “Anagram А enc 1 ‹ dae 
Approach Rorschach | Function П Design Design Rating 
Ё И ЕЕ > == 2 =. 
Когзсһасһ = 0g +03 | == = E Bi |" di 
Function „26 -54 3 20 13 E: E. 
Anagram I —.36 —.13 —5,32 oo —.20 27 
v I S 22 
8 — +22 
Anagram II -09 | — +20 cio _ E: m 
Stencil Design —.25 —.18 m = 35 Bi. 
Block Design об | +12 ET 
“| اا‎ a ae 
Total Rating —.07 | —,13 —.02 = 2 = gi 3 
SS ERE — ИЙ 
in Rigidity will rate low in Whole Whole-Part Approach, and on Fle 
Approach. This result is 


in line with 
clinical findings on the Rorschach Test, 

The results indicate, then, that the un- 
Structured tests show Flexibility-Rigidity 
Process and Whole-Part Approach as two 
Separate processes, while the more struc- 
tured tests seem to use simil 
as evidence for both \ 
proach and Flexibility-Rigidity Process. 

The Function Test shows a correlation 
of .54 between scores for Whole-Part Ap- 
proach and Flexibility-Rigidity Process. 
The Function Test has been shown 
earlier to be a test that seems to measure 
different behavior from the other tests. 


ar behaviors 
Vhole-Part Ap- 


SUMMARY AND IMPLICATIONS 


Review of the Plan of the Study 

A battery of six tests involving verbal 
or perceptual problems, and varying 
from less structured to more Structured, 
was administered to a group of 19 college 
students. This Broup was a highly homo- 
geneous population with respect to age, 
amount of education, total and subscores 
in the ACE test, reading comprehension 
test scores, and writing skill test scores, 

The individual test records were ana- 
lyzed to determine the subjects’ scores on 


bility-Rigidity Process, 


Summary of the Results 
Whole-Part Approach 


The Broup showed variability with = 
spect to Whole-Part Approach in all six 
test situations, Correlations between ans 
test and the total score (see Table 4) нас. 
Significant for the Rorschach "Test anc 
the Stencil Design ‘Test at the 190 leve 
and for the Anagram I and the E 
Design Test at the 5% level. The results 
indicate a fair 


istency of 
degree of consistency 
Whole-P 


кыйшай. 
art Approach for the individt 
Flexibility. Rigidity Process 
ЖҮ, | 
The Broup showed variability W 
respect to Flexibility-R igidity Process 
all of the Six test situ 
The corre] 
with the tot 
significant 
These тези] 


ations. , 
ations of three single е 
al score yield results that 
at the 595 level or pend 
ts indicate a moderate degre 
of Consistency of the Flexibility Rigi) 
Process. However, pes 
correlation 
Scores on 


there is no signifi em 

between Flexibility-RigiditY 
an unstructured and a ae 
tured test, On the other hand, correla 
tions among the unstructured tests sho! 
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ignificant correlations, as do also correla- 
tions among the structured tests. This 
would indicate that Flexibility-Rigidity 
Process is fairly consistent for the indi- 
vidual on tasks that are similar in struc- 
ture, but not consistent for tasks that dil- 
ler in structure (see Table 5). 


Relationship of Whole-Part Approach to 
Flexibility-Rigidity Process 

The correlations of Whole-Part scores 
and Flexibility-Rigidity scores 
Unstructured tests are not significant, 
While the correlations among structured 
tests are statistically significant. This 
Would indicate that. the unstructured 
tests show Flexibility-Rigidity Proce 
апа Whole-Part Approach as two separa- 
rate processes, while the more structured 
tests seem to use similar behaviors as 
Evidence for both Whole-Part Approach 
| Flexibility-Rigidity Process (sce 

l'able 6). 


among 


S 


Conclusions 
_ The findings of this study seem to 
к the following conclusions: 

. Problem-solvi ing processes can be in- 
а from observation of problem-solv- 
b: behavior in special tests. 

sA Broup of subjects homogencous as 
intelligence shows individual diller- 
Cnces with respect to Whole-Part Ap- 
Proach and Flexibility-Rigidity Process. 
3. The Whole-Part 
кү) consistent process for the m: ajority 
1€ subjects in situations that differ 


to. 


Approach is a 


as to content and degree of structure. 

Flexibility-Rigidity is a fairly con- 
sistent process for the individual on tasks 
that are similar in structure, but not for 
tasks that differ in structure. 

5. Whole-Part Approach and Flexi- 
bility-Rigidity are two separate processes 
when observed in less structured situa- 
tions; in more structured situations the 


two processes are more closely related. 


Implications 


Problem solving has mainly been looked upon 
as purely intellectual. behavior. However, the 
Whole-Part Approach and the Flexibility-Rigid- 
ity Process are. variables that have often been 
related to an evaluation of personality charac- 
teristics. Thus, an investigation of problem- 
solving proc may reveal how the individual 
reacts in certain. problem-solving situations, and 
it might lead to other measures of "functioning 
or effective intelligence." 

A focusing on process makes it possible to 
make finer differentiations of intellectual func- 
tioning among individuals at the superior level 
of intelligence. The differences are then not due 
to the fact that a subject solves one more prob- 
lem than the other subject, but rather to the 
problem-solving processes used. 

The investigation of problem-solving proc- 
s also leads to an explanation of why sub- 
fail to solve certain problems. "This in- 
formation is usually not available if one deals 
mainly with the end products of the problem- 
solving activity, Опе has then to be content to 
state that the subject cannot solve analogy prob- 
lems, numerical problems, ete. 

It should also be pointed out that the prob- 
lem-solving processes which have been identified 
lend themselves to cutting across content cate- 
gories. Whole-Part Approach and Flexibility- 
Rigidity are processes that can be identified in 
problems of a verbal, spatial, or numerical na- 
ture. 


APPEND 


IX 


BEHAVIORS IDENTIFIED IN EARLIER INVESTIGATIONS 
AS RELATED TO WHOLE-PART APPROACH 


Whole Response 


Sargent (13)—Looking at letters with no attempt 
to form syllables or combinations, 
—A rapid solution in the first few sec- 
onds. 


Rorschach (11)—Responds to area as a whole. 


Hanfmann (7)—Formulating hypotheses in thinking 
without much paying attention to 
blocks. 

—Handling and moving of blocks is 
reduced to a minimum, 


Rapaport (10)—Passive speculation, with little or 
no manipulation of blocks. 


Goldstein (6)—To plan ahead ideation: 
—To abstr: 
flectivi 
cepts, 
—To hold in mind simultane 
ous aspects, 


ally. 
act common properties re- 
ely, to form hierarchic con- 


ously vari- 


Part Response 
i i i otters without 
—Forming combinations of letters with 
any definite plan. . 
—Secking a common prefix or suffix. 


rel-consonant 
—Making usual consonant-vowcl-consonan 
combinations. 


1 7i А с 
—Trying one letter at а time with all th 
others, 


—Combining letters at random. 
—Solution emerges slowly in steps. 


—Responds to part of the area. 


in constant touch with material. 

s ideas from looking at blocks and han: 
dling them, Hus 

—Has groupings first and formulates prin 
ciples afterwards. 


—No attempts are made to organize impres- 
sions into generalizations. 

—To Concentrate on a single aspect of а 
situation, he 

—Thinking and acting are directed by me 
immediate claims which one particular 


ar-world 
aspect of the object or of the outer-wor 
makes, 
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Effects of Task Motivation and Expectancy of 
Accomplishment upon Attempts to Lead’ 
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Тик PROBLEM 
peo experiment is the third in a 
series conducted to. test. hypotheses 

about the effects of situational and per- 
Sonal variables upon the frequency of 
attempts to lead. The first experiment 
examine 8 . и 
amined the relationship between pos- 
Sessi 3 top " 
n of relevant information and the 
re a MN 
“quency of attempts to lead (3). The 
sec х 

econd experiment tested hypotheses 
concern} s à 

ЕТШЕ the effects of (а) needs [or 
achieve рт n 
К Nevement and affiliation, and (b) posi- 
ive ; 8 3 

Ve and negative reactions of other 
on 
Te Members upon attempts to lead 
ye Thre ; 
a Ihe present experiment tests hy- 
20 Wi . > т 
ti theses about the effects of two addi- 
lon; TTA А N 

nal variables: task motivation and 
c NE 

ey 
for Tue authors wish to thank Delos Wickens 
brews Cooperation in securing subjects. Ap- 
ds $ tation is due to the students who participated 
j СЕ in the experiment and to Mrs, Ruth 
Kathe ME. Mrs, Sadie H. Richie, and Mrs. 

lerine Mulligan, who rendered invaluable 


Cleri 
ран Ssistance either іп the conduct of the 
Scri "Ment or in the preparation of the manu- 
Script, 


Noon’ research was conducted under contract 

7 Г.О. III NR 171 123 between the Ohio 
University Research Foundation and the 
bre E of Naval Research, The opinions 
Paper К are those of the authors alone. This 

5 à condensed revision of a technical 
aring the same title submitted to the 
Of Naval Research. Sample copies of forms 
ш Ше study are appended to that report, 
Stake. ау be obtained on loan from The Ohio 
Diversity library, 


State 


expectancy, The hypotheses involved in 
each of the three experiments are based 
upon a tentative theory of leadership in 
small groups (3). 

In the tentative theory the following 
statements relevant to the present experi- 
ment appear: 


The motivational variables that determine 
whether a given individual will attempt a given 
leadership act lic in two major arcas, The first 
involves. disposition variables, Of these 
variables the most central to our problem in- 
clude (1) the strength of his dissatisfactions 
associated with the mutual problem, (2) thc 
strength of his social nced-dispositions that the 
group has a potential of satisfying. The second 
area involves cognitive estimates or expectations 
of the individual concerning the probable con- 
sequence of the act of leadership he is consider- 
ing attempling. Two important areas of variables 
ated to the estimates concern (1) the proba- 
bility of solving the mutual problem, and (2) the 
probability of increasing or decreasing the po- 
tential of the group for the satisfaction of social 
necd-dispositions. We may express the proba- 
bility of an individual's attempting a leadership 
act as function of (1) his estimate of the 
probability of the act leading to mutual problem 
solution, (2) his estimate of the probable effect 
of the act on the potential of the group for 
need-disposition satisfaction, (3) the degree of 
dissatisfaction he feels with the mutual problem, 
and (4) the strengths of his relevant social 
need-disposition (3, p. A-26). (Italics do not 
appear in the original but are used here to 
emphasize the most relevant statements.) 


a 


Three hypctheses, which are related 
to the above excerpt [rom the theory, 
were formulated: | 

I. Individuals who perceive that the 


Bu: 
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solution of a mutual problem will be 
more rewarding attempt leadership with 
greater frequency than individuals who 
perceive less reward associated with the 
solution. 

IL. In a situation in which acts of 
leadership cannot result in a solution of 
the mutual problem, individuals attempt 
leadership with less frequency than indi- 
viduals who are in another situation in 
which a relationship may exist between 
acts of leadership and the solution. 

ПІ. Interaction between these two vari- 
ables is insignificant in its effect upon 
the frequency of attempted leadership. 


PROCEDURE 
General Design of the Experiment 


The experiment involved two inde- 
pendent variables, task motivation and 
expectancy of accomplishment, and a de- 
pendent variable, frequency of attempted 
leadership acts. Both task motivation 
and expectancy of accomplishment were 
varied at two levels, Each of the 24 
groups began the experiment under sim 
lar expectancy conditions, 
of the groups worked under conditions 
of “high” task motivation, while the 
other half worked under conditions of 
“low” task motivation, Midway in the 
experiment, expectancy conditions were 
altered so that one-half of the “highly” 
motivated groups worked under condi- 
tions of “high” expectancy, i.e., condi- 
tions under which relatively large accom- 
plishment was possible, The second half 
of the “highly” motivated groups con- 
tinued the experiment under conditions 
of “low” expectancy in which, if the situ- 
ation was analyzed objectively, 
possible to accomplish the task. The 
groups working under “low” Motivation 
also were divided into "high" and “low” 


expectancy groups for the second half 


i- 
but one-half 


it was im- 


of the experiment. Thus, the experiment 
was conducted in two sessions, the second 
following immediately after the first. = 
sion I provided a situation in which the 
two conditions of task motivation could 
be established and their effects observed 
without complication. Session H pu 
vided an opportunity to study the effects 
of both task motivation and expectancy 
and also their interaction. x 

The research design also provided me 
(a) independent checks upon the degree 
to which the two independent variables 
actually were manipulated, (b) poeni 
tion of the frequency of attempts to ee 
and (c) control of extraneous facto 
likely to influence attempts to lead. де 

The remainder of this section of t Š 
report is devoted to detailed ge 
of the selection of the subjects, са 
Manufacturing Problem, the [белй 
used to vary task motivation and eek 
tancy, and the procedure used to or 
upon the establishment of the des! 
experimental conditions. 


Subjects 

nrolled 
rse 
was 


Subjects were selected from the men УШ 
as students in the introductory psychology ^^ у 
at The Ohio State University, Each subjecit 
able to fulfill part of his course Re оја 
by taking part in this gapenn eo. was 
ranged in age from 18 to 26 ye a1 with 
exercised to avoid including individua? 
obviously deviant physica] characteris ей to 

The 96 volunteer subjects were a other 
24 four-man groups without reference | nce and 
criteria than those of scheduling convene’ ted 
the assignment of previously unacquaim 
dividuals to the same group. 


The Manufacturing Problem 


оте û 
Теп Preliminary groups were run e ying 
Satisfactory Procedure was devised for was the 
task motivation, The major problem тоу" 
Creation of a condition of “Jow” task pur 
Чоп, This difficulty was due in pu 
decision to utilize in this experiment s 
tion of the Manufacturing Problem ( ji: 
Was used as the experimental task in the P 


odific® 
which 
vious 


MOTIVATION, EXPE 


experiment, and which possesses a high degree 
of intrinsic interest. Since the procedure for vary- 
Ing t motivation is closely bound to the nature 
of this task, a general description of it will be 
Siven at this point. 

The Manufacturing Problem is essentially an 
elaborate construction task, The task requires 
cach Broup to organize its members for the pur- 
pose of constructing toy models from ‘Tinkertoy 
Parts In а manner to realize maximum credit 
Bains, The groups are called upon to engage in 
discussion, planning, assembly, 1 soning, and 
arithmetical computation, This task incorporates 
the characteristics of a number of the tasks fre- 
quently used separately in small-group research. 
done gniset, 1 1 h group is given cither three 
"lod in cash ( high . motivation) or 300 points 
ron Оне) with which to begin work. 
models din only for toys exactly like the 
йел ae ayed. Each group member is pro- 
liemies te a booklet of information, which 
Points) « 1C cost (in terms of either moncy or 
[Б be = the group of separate Finkertoy parts 
amoiint ree from the supplier, and gives the 
Sigh ‘toy A credit to be earned by completing 
experia The subjects are informed that the 
Out th, J includes two sessions. It is pointed 
cost Be after the first work period both the 
ЙӨНЕ. the credit schedules, as well as the 
the dile һе gained. from the construction of 
Minute ы toys, will be changed, Each 20- 

Vork session is preceded by a 5-minute 
period; the first work session and the 
riod are separated by a very 
ak. A clock above the buyer's table is 

Into successive intervals that com- 
{wo planning periods and the two 
ions, 

Cr to secure parts, the group is required 
t Special order forms, have them counter- 
y all group members, and submit them 
CXperimenter, After the instructions are 
Operation шт pari the п ees 
hands oa пе group is left complete y in the 
ignated he members, xcept that during the 
Struction of planning periods no actual con- 
however u ду. is permitted. It is emphasized, 
nas ts пас discussion and revision of plans 
nited to the planning periods but can 


enga ; 5 A E 
Session Ecd in at any time during the work 


In ord 
9 fill ou 
Signed p 
to the 


vins ч. рһузїса1 arrangements for the task in- 
rae use of four tables placed around the 
functions” These tables are identified by their 
Wan tit 5: (а) a supply table, where a large 
Wailapy, and variety of Tinkertoy parts are 
arge rth Ih appropriately labeled boxes; (b) a 
Scratch € upon which are placed various forms, 
Com ads, and pencils, and upon which five 

Tinkertoy models are displayed (a 
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"top," а "man," an “airplane,” а "wagon," and 
a "ladder"); (c) a table to which finished tovs 
are brought in order to receive credit; and 
(d) a table upon which copies of current maga- 
zines are placed. 

In the earlier experiment (6) this task proved 
to have many desirable characteristics as а 
mutual problem for men college students, Among 
these characteristics are: (а) a wide у; riety of 
interlocked or dependent subtasks permitting or- 
ganization, division of labor, and utilization of 
individual differences in ability; (b) high intrinsic 
interest value, probably introduced by the close 
similarity to "running a business"; and (с) a 
direct measure of quality of group performance, 
às indicated by the amount of profit earned. 


Procedure Used to Vary Task Moti- 


vation 


Under both the "high" and "low" task motiva- 
tion conditions, the task has the following fea- 
tures: 

(a) There are two work sessions, each con- 
sisting of a 5-minute planning period and a 
20-minute work period. , 

(b) Tinkertoy parts from which model toys are 
constructed are secured from the experimenter. 

(c) The group members decide which of five 
toys to construct and how many of each to make. 

(d) Completed toys are turned in to the ex- 
perimenter for credit. 

(c) There is opportunity for division of labor 
among the members of the group. 

(f) An account is maintained of the resources 
("capital") currently available to the group for 
the conduct of its operations. 

(g) There are no restrictions on free com- 
munication among members of the group. 

(л) Identical materials are available with 
which to construct toys, and the arrangement 
of the laboratory is identical. 

The differences between the task as presented 
under the "high" and the “low” task motivation 
conditions pertained to (a) the nature of the 
reward, (b) the attitude that the experimenter 
displays toward the task, (c) the meaning of thc 
task, and (d) the amount of routine involved in 
securing parts, To facilitate comparison of the 
conditions, these different procedures are ar- 
ranged on following page in parallel columns. 

The conditions of “high” and "low" task 
motivation were established by changing only 
those task characteristics that subjects, in the 
preliminary runs, reported to be sources of their 
interest in the task. Care was exercised to pro- 
vide under both conditions equality of oppor- 
tunity to attempt leadership insofar as this vari- 
able was independent of task motivation. 


Reward 


Experimenter 
Attitude 


Meaning 
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High Motivation 


Subjects worked with money (coins and 
bills) and retained all the profits their 
group made. 


Experimenter gave the instructions with 
enthusiasm. He indicated that the ex- 
periment was very important and that 
it involved money supplied by the 
U.S. Navy. After the experimental in- 
structions were read, he ignored the 
magazines on the table and proceeded 
in businesslike manner to his station as 
“buyer” and supplier." 


The task was presented as a simulated 
manufacturing enterprise and as а 


Low Motivation 


Subjects worked for "points." No coins 
or bills were used, 


Experimenter gave the instructions 
without enthusiasm, He indicated Dy 
side remarks that the experiment prob- 
ably would be discarded. He suggested 
that the subjects might find the task 
dull and uninteresting. After giving 
the instructions he proceeded to read 
the magazines, 


The task was presented as a Ша 
; ; sianet 
construction task originally designe 


ability. 


The routine or “paper work” 
ated with ordering supplies w: 
mized by supplying parts in 
“kits.” Only one order form was re- 


Routine 


quired, 


very good test of individual and group 


15 mini- 


э К А i 
for use with school children but no 
appropriate for college students, 


associ- 

Parts had to be secured separately (n0 
kits). Order forms were required in 
triplicate, 


lots or 


Procedure Used to Vary Expectancy of 
Accomplishment 


Since the prime objective of the task was to 
make profit (or gain points), it was possible to 
vary the expectancy of accomplishment by chang- 
ing the relationship between the cost of parts 
and the selling prices of completed toys, Under 
conditions of “high” expectancy it was possible 
for group members to make a relatively large 
cash profit (or many points) but under condi- 
tions of “low” expectancy, it was not, However, 
the success of such a procedure is likely to depend 
upon whether the “objectiv situation is cor- 
rectly perceived by the subject, Whether accurate 
perceptions will occur depends їп turn upon two 
factors: (a) the clarity with which the necessary 
outcome is presented to the subjects, апа (b) the 
strengths of motives that may influence subjects 
to misinterpret the facts as presented. The results 
of recent studies (e.g. 1, 2) of the rel 
tween personality factors and pe 
tion suggest that we might expect to find many 
failures to perceive the fact that no profit could 
be made under “low” expectancy and "high" task 
motivation. If present, this tendency would intro- 
duce an interaction effect between the two main 
variables of the experiment, In order to mini 
mize the possibility of an interdependence of 
task motivation and expectancy of accomplish- 
ment, an effort was made to maximize the clarity 


ation. be- 
reeptual distor- 


for the Subjects of the profit, or point-gar" 
Situation, Toward this end the experimenter * 
instructions to the subjects emphasized ШЕ 
portance of determining the possibilities for ga " 
in advance, and illustrated. in detail the jon 
cedure for determining profits (or. point B ed 
from the cost mg 


а ice i mation 
Ind price informat the 


Fable 1 shows the difference between 


3 ped 10 

points. (or moncy) the subjects surrendered P 
PH " ies Fa i S Ё ey 

obtain Supplies апа the points (or mon у) the 


experimenter credited to the group пров 
completion of cach 10у. 


TABLE 1 


siox T: 
, HIGH 


Session IT І 


Article Session 1 High Ex- Low m 
9 каг yectanc? 
چ‎ pectancy Р oa 
Man лс ext 
à —1 
in ? 26 : 
Airplane 15 yi —4 
agon 32 8 о 
Stepladder =й igo E 
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It can. be seen from Table а that the group 
Member who correctly interpreted the data sup- 
plied him and proceeded accordingly to work on 
the task could expect to. gain some profit or 
Points during Session I. Furthermore, during Ses- 
Sion TH, if he were working under the high expec- 
taney condition he would note that on three of 
the five toss a more attractive gain was possible 
than he had encountered during his first session. 
Under Jow expectaney, however, the situation 
would be accurately perceived as ollering no op- 
Portunity to gain profit or points by working on 
the task. 


Procedure for 
Leadership ets 


Observing Attempted 


Te Manual, reproduced elsewhere: (3), estab- 
ned rules for differentiating attempted leader- 
"P acts from other behavior of group members. 

y- 


vise, observers located behind 4 one-w 
mirror made a tally mark in an appropri- 

Me column of a prepared form cach time a group 
ember made an imperative statement specifying 
epe е in the immediately present mode of 
Stoo ee of the group members during the 
1 с S of mutual problem solution" (3. p. A«40). 
ad" procedure ol tabulating attempted lead 
the | S Was utilized without modification in 
AN experiment, Since an estimate of the 
Deen iit of agreement between observers taa 
was Же. ча on three other occasions (з. 6, 7) ane 
Wile tee high, no further check of this kind 
med necessary, It is sullicient. to state 

Pairs 2 Median intraclass correlation within 
rained observers is above 4) and ranges 


from 80 to 9б 
One танро, - 
Serves, "ther point should be noted. Two ob 


S Were used in this experiment. One ob- 


Serve, 
ri p, et available only for thosc experiments 
Noon " te morning, the other only in the after- 
the x h order to guard against gantaunding 
differen imental variables with ШОШЫП 
recordet, in the average number of ts they 
balanced the experimental design was — 
Squat wi SO that cach condition w repres anne 
Y dn the morning and afternoon runs. 


Ch echs 
Pecta ney 


on Task Motivation and Ex 


Ty, " " ^ 
wo Checks upon task motivation were built 


s !€ experiment, The first was provided by 
А Ubjects’ responses to an item included in a 
Was: OSCEX periment Questionnaire, This item 


important was it to you that your 
" Чо an outstanding job on the task? a. Ex- 
У important, p, Fairly important. с. Not 
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very important, d. Rather unimportant. e. De- 
finitely unimportant, 

The second check was provided by the ratings 
of an additio observer (located behind a onc- 
way-vision mirror) who at the end of each five- 
minute interval rated the groups on six items 
of task motivation, These items were as follows: 

1. How enthusiastic was the group? 

2. How hard did the group members work? 

3. How tense did the group members appear 
to be? 

t How important did doing well on the task 
appear to be for the group members? 

5 How intent were group members upon 
making points or profit to the exclusion of other 


5 


things? 

6. How motivated in general were the mem- 
bers of the group to make points or profits? 

Although the observer usually rated group 
rather than individual task motivation, he was 
instructed to rate individuals separately when- 
ever he perceived dillerences in motivation among 
the members of the group. 

Thus, as a check upon the procedure used to 
achieve high or low task motivation in the ex- 
periment, we have (a) the individual's subjective 
report about his own task motivation and (b) the 
judgments of an outside observer about task 
motivation, 

An item also included in the Post-Experiment 
Questionnaire was designed to provide a check 
on the procedures used to establish the ex- 
pectaney conditions, This item was: 

During the second work session, how difficult 
did vou find it to do well on the К? a. Im- 
possible to do well. b. Relatively dificult to do 
well. c. Don't know. d. Relatively casy to do well. 
e. Very easy to do well. 

In the experimenter's view his procedure estab 
lished an unambiguous difference between the 
high and low probabilities of accomplishment. 
But there remained a question as to whether the 
individual subjects under low expectancy would 
consider themselves to be "doing well" with ref- 
erence to other criteria than their performance 
on the official task. 


RESULTS 
The degree to which the desired ex- 
perimental conditions were established 
will be examined first, after which analy- 
ses of the effects of the independent vari- 
ables upon the dependent variable will 
be presented. 


Checks Upon Experimental Conditions 


In order to assess the degree to which 


HEMPHILL, PEPINSKY, KAUFMAN, AND LIPFTZ 


TABLE 2 | 
е SRUTRE ТЕН. “How 
R SES OF 96 GROUP MEMBERS TO THE Posr-EXPERIMENT QUESTIONNAIRE ITEM, I 
ESPONSES OF 


Important Was Ir то You THAT Your Group Do AN OUTSTANDING Jon ON THE 
MP 1 ' 


Conditions 
Motivation Expectancy Combined 
Response Category (M) (E) ا‎ = 
К High M High M Low M Low y 
High | Low | High | Low ШКЕ ШЕМ Low E 
mod 
8 10 6 | i 
. Extremely Important 28 7 24 11 1f TE 
[^ Fairly Important 20 27 18 29 6 14 Ё à 
c. Not Very Important o 12 6 6 o o | z ( Е 
а. Rather Unimportant о о І о о 
е. Definitely Unimportant o 


o I o o 9 | я 


experimental procedures produced the 
desired conditions it is necessary to an- 
swer two questions: (а) How successful 
were we in establishing high versus low 
task motivation and high versus low ex- 


pectancy? (b) Did the variations which 


were introduced for the purpose of creat- 


ing high and low €xpectancy interact in 
апу way with the task motivation of the 
subjects? The answers to these questions 
are provided by the subjects’ responses 


to the Post-Experiment Questionnaire 
and the observers’ ratings made during 
the experimental] session. 


Motivation 


As a check upon the manipulation of 
task motivation, an analysis was made 
of the responses to the item of the post. 
Experiment Questionnaire that 
cerned the importance of q 
standing job on the task, Th 
are presented in Table ». 

Chi-square tests were made of the as- 
sociation between experimental condi- 
tions and the frequency of the responses 
in the five categories. Chi square was 
partitioned (4) into three components: 
(a) that associated with the high and low 
motivation conditions, (b) that associated 
with the high or low expectancy condi- 
tions, and (c) that associated with the 


con- 
oing an out. 
ese responses 


interaction. of motivation and expect- 
ancy. The total chi-square value is P 
(significant at the .o1 level), which "i 
vides into the components 27.64 (sign! К 
cant at the .о1 level) associated with on 
tivation conditions, 7-42 (not significant) 
associated with expectancy сеси 
апа 5.91 (not Significant) attributable 2 
the interaction of the nid, pane 
ables. The subjects’ reports are crn 
with the experimentally created motiv‘ 
tion conditions, 1 

We note, however, that only s S = 
48 subjects run under low Ee 
choose the responses “rather шаш 
tant” or “definitely unimportant.” It de 
pears that the 48 subjects cannot aj ny 
Scribed as having low motivation in 2 T 
absolute sense, but rather should us ~~ 
sidered as being relatively less motiva m- 
than the highly motivated group = 
bers, 

A second independent check uper = 
creation of high and low motivation “the 
ditions is provided by reference to 


es à ver. 

ratings made by an outside observe 

e Sosa youps 

1 the 
jn- 


*Two Observers were used; one rated B 
Tun in (he morning, and the other ас 
afternoon groups. The observers were "ditions 
formed of the specific experimental com ds 
Under which each group they observed Y 

© run. However, it is likely that they ™ 
carly guess concerning these conditions. 
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| the end 


each five-minute interval 
during the experiment an observer rated 
the group on six items relevant to task 
motivation. The observer was instructed 
to rate the entire group as a unit, but if 


any one or more group members could be 


of 


differentiated from the others, to rate 
those members as individuals. Although 
Most ratings were made for the groups 
as units, some individuals were differen- 
tiated. When no differentiations are 
| made among group members, the group 


rating is assigned to each of the [our 
Individuals, 

A test of the dillerence between the 
Means (based on the sum of all ratings, 
t€, six items rated at the end of each of 
10 five-minute periods) of the high and 
low motivated groups yields a value of 

9-42 (significant at the .o1 level). Clearly 

the observers’ ratings correspond to ex- 

perimenta] variation 
| Conditions. 


іп motivational 
In order to examine the degree of cor- 
Tespondence 
Tatings of 

i tion 


between the observers’ 
a group member's task motiva- 
i Mois the group member's estimates 
tion P IPortance of the task, correla- 
by eius computed between the ratings 
te observers and an index based 
"pon the responses by group members 
| 10 the first item (“importance”) of the 
SCEXperiment Questionnaire. This in- 
te Was constructed by assigning to the 
“SPonses to the Post-Experiment Ques- 
tionnaire values of one to five points 
OWest to highest rated importance). 
б 3 presents һе correlations be- 
rain each of the six items and the total 
185 both by session and for the two 

Sessions combined. 


aie Of the six items is correlated positively 

he Subjects’ estimates of the importance 
to pog Well on the task. The item that appears 
the « MOst strongly related is, "How tense did 
Broup members appear to be?" These correla- 


[d 


tions are до for the first session and 
second. Two items—“How hard did the members 
work?” and "How intent were group members 
upon making points or profits to the exclusion of 
other things?"—tend to be less strongly related 
to the subjective estimates, During Session II, 
the correlations between these latter two rating 
items and the subjects’ estimates (39 and .40) 
are both significantly less (.05 level) than for the 
first cited item ( Both these items reflect 
the level of task-oriented activity of the group 
members. The first item expresses “tension” or 
anxiety displayed in the situation, Perhaps the 
difference in the correlation coefficients is to be 
explained by the behavior of individuals who 
quit work under high motivation and low ex- 
pectancy, but who, nevertheless, exhibit other 
signs of their high motivation. If the item, 
"How hard did the group members work?" is 
excluded from the total motivation rating, some 
of the effect of amount of activity as an indica- 
tion of motivation may be removed. Then the 
correlation between the observers’ ratings and 
the subjects’ estimate of the importance of the 
task becomes (instead of .53 for all six items), 


; for the 


These checks on the procedures used 
to vary task motivation show that (а) 
the subjects attach less importance to 
doing an outstanding job on the task 
under conditions of low motivation than 
under conditions of high motivation, (b) 
outside observers rate motivation higher 
under conditions of high motivation 
than under conditions of low motivation, 
and (c) the subjects апа the observers 
tend to agree about the subjects’ motiva- 
tion to work on the task. It can be in- 
ferred, therefore, that the desired moti- 
vational conditions did exist for the sub- 
jects during the experiment. 


Expectancy 


The experimental conditions of high 
and low expectancy were introduced dur- 
ing the second work session only. High 
expectancy was produced by presenting 
the subjects with more attractive profit 
or point schedules than those used dur- 
ing the first session. Low expectancy was 
established by schedules so constructed 
that the subjects could not realize a 
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TABLE 3 


U 
MOTIVATION MADE BY A 
(N=g6, Mean Index = 


Rating Item Correlated with the Questionnaire I: 


adex 


‘SES OF GROUP MEMBERS TO AN [TEM ON 
TING TASK MOTIVATION AND RATINGS OF 
N OUTSIDE OBSERVER 


8, SD Index =.77 


Mean SD 7” 


First Session 

How enthusiastic was the group? 

How hard did the group members work? 

How tense did group members appear to be? 

How important did doing well on their t 
for group members? 


ask appear to be 


18.2 4.82 37 
21.5 4.00 jt 
14.2 5.84 4 


б 
ч 25 531 d 
How intent were group members upon making points or 
profit to the exclusion of other things? 27.7 3.43 +7 
How motivated in general were members of the group to ` 
make points or profits? 21.0 dos .32 
Sum of the above items. 128.3 20.42 дю 
Second Session 
How enthusiastic was the group? is 40 
How hard did the group members work? 1878 30 
How tense did group members appear to be? 15 E «88 
How important did doing well on their task appear to be ай: 
for group members? 8 i 47 
How intent were group members upon making points or bus БЯ 
profits to the exclusion of other things? | ? 49 
How motivated in general were members of the group te шын pue 
make points or profits? p 48 
Sum of the above items. 10.6 9.88 40 
| к j 101.0 52.00 
Session I ё II Combined 
How enthusiastic was the group? $1 
How hard did the group members Work? 33.5 En 
How tense did group members appear to be? 39.9 54 
How important did doing well on their task " 27.8 ` 
be for group members? "SS appear to m 
How intent were group members | 5 4 5 е 
A 2 $ upon mak ine 44.1 13.1 
profits to the exclusion of other things iie Points or a HW 
How motivated in general were ; Si 
k a members 6.49 hi 
make points or profits? ers of the group to ag E 6 
Sur Pd еї 3 4 
Bum of the above items, 37.6 13.605 43 
"s RI IG. T 220.0 68.04 
* 4 elati. H . " Барына 0.0. 
A correlation of 727 15 significant at t 


monetary or point gai 
the task. Were the percepi 
: а stent wi PAIN 
perimenter's intentionsz 
Question II on the Post-Experi 

Questionnaire reads, “During the a 
work session how difficult did yo, Pis 
to do well on the task?” q, Imp, DR Hs 
do well. b. Relatively difficu]t ane t 
c. Don't know, qd, Relative] Hoe wl 
well. e. Very саѕу у easy to do 


10 do 
- Ww 
This item was deliberately гы 


avoid a direct question abe; «d Е 
aking 


he Ке level, 


jd 
"E 

ч ect > 

prati OF points" so that the subje ir he 
ree " sponso ns 

eC 00 select a d or ¢ respon yer 
should in ES 


define “doing well 
an points or profits. 51 
oi oreet 
as anticipated that sub] and * p^ 
: the task of the pio: har ' 
Unue to work for other motive? f 
Make pr ; ints К 
паке Profit (or gain points). «tio? 
Table 4 represents the distr! : {хр 
Кач ^osU 
"éSponses to this item of the Ро 5 
me ; : i 
ent Questionnaire. TE 
Chisquare tests were made ‹ маі © 
SoCiation between the ex perime 


other th 
bility w 
redefine 


“phe 1 


„е —— ——————— 


MOTIVATION, ENPECTANCY OF 


ditions and the lIrequency ol the subjects’ 
responses to cach of the five categories. 
Chi square was partitioned into three 
components: (а) one associated with the 
high and low task motivation conditions, 
(b) a second associated with the high and 
low expectancy conditions, and (c) the 
third associated with the interaction ol 
the motivation conditions and the ex- 
Pectancy conditions, The total chi square 


Wi eee 
5 found to be 99.45 (significant at the 


ACCOMPLISHMENT, AND ATTEMPTS TO LEAD ¢ 


The Effects of Task Motivation and of 
Expectancy Conditions Upon the 
Frequency of Attempted 
Leadership Acts 

Two analyses of variance were per- 
formed in order to study the effects ol 
task motivation and expectancy upon at- 
tempts to lead. Before these results are 
presented, descriptions will be given of 
the transformation. procedure employed 
to convert. the observed frequencies. of 


TABLE 4 


Dist 
ISTRIRUTION OF THE RES 
TEM, “DURING THI 


ECOND S 


R , Motivation 
Sesponse Category | (M) 


High | Low 


eer. | 

a, Р -—— سڪ‎ = — 

b possible to do well | о | 7 

С: Donte УУ difficult todo well 0 | 4 

d. Rance Ж 

е dem Wely casy to do well | 15 ži | 
У easy to do well | n | 0 


n eat which divides into а compo- 
7-80, associated with the expect- 
conditions (significant at the .o1 
"а component 7.1 associated with 
Motivation conditions (not signifi- 
59 associated 


ancy 
level) 
the 

Cnt) 


With ps a component 4.53, is 
Expec мы Interaction ol motivation and 
“ancy (not significant). 

It Should be noted that 23 of the 48 
ic, ехресапсу subjects indicated ш 
seine either "very easy ge well, or 
the ` у easy to do well." In view of 
they ‘i estive _condiuons under which 
ксб Worked, it appears that these sub- 
eis noe have defined “doing well in 
i : other than points or profit Бану 
su Sener), however, the responses ol the 

Jects to this item are in accord with 


the 7 М 
requirements of the experiment. 


SNSES OF 96 GROUP MEMBE | Z 
siox, How Dirricerr Dip You Fixo Ir 


ro THE Post 


Conditions 
Expectancy 
(E) 


ch | Low High M|High M Low M Low M 
High | Low High E Low E High E Low E 


Combined 


1 15 1 | 8 | o | $ 

8 5 6 | 3 2 2 

3 5 o 1 3 | 4 
20 13 10 | 8 16 | 5 

10 10 7 4 3 | 6 
attempted leadership acts into “at 


tempted. leadership scores, and of the 
pattern of the analyses. After the general 
plan of analysis has been made clear, the 
results of the two analyses of. variance 
will be presented. Finally, certain. fur- 


ther analyses will be made of the more 
significant ellects revealed by the main 
analyses. 


Attempted Leadership Scores 


Vhe data of the attempted leadership scores 
are the observers’ tallies of the number of at- 
tempted acts of leadership during specific inter- 
vals of time. The frequencies of the occurrence of 
ane nt observed within a limited time interval 
approximate a Poisson distribution, The square 
root transformation has been recommended (5) 
for the purpose of stabilizing the variance ol 
these types of data. Since homogeneous within- 
group variance is required for appropriate use 
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TABLE 5 
7 AND TRANSF Ера ATTEMPTED LEADERSHIP SCORES 
JTION OF RAW AND TRANSFORMED® £ 
DISTRIBU (ween cR 
Session I Session II 2 
1 Transformed "requency 
Raw Score Маш голе Frequency Raw Score ا‎ Freq ت‎ 
o 
1 41- 9 
I- 9 : 
32-40 8 2 32-40 8 : 
25—31 7 9 25-31 7 8 
18-24 6 8 18-24 6 ^ 
E і : pr А 19 
[m 4 25 8-12 H 18 
57 $ 19 5-7 3 г 
m 2 12 2-4 2 i 
x х 5 1 1 з 
9 2 3 o o p ы 
in each cas i " 33: any value be- 
^ These transformed scores are in each case the greatest integer less than Мзхреш., any У 
tween 7.00 to 7.99 received the value 7. 


of analysis of variance, “attempted leadership 
Scores” were computed based оп à square root 
transformation of the frequency data, 


Table 5 presents the Separate distributions 


of the original and the transformed (Мәх) data 
for Sessions I and II, 


Chi- 
tributions of transformed 


obtained values of chi Square are 5.23 for Ses- 
sion I (p> +5) and 9-13 for Session IT (p > .2). 
A basic assumption underlying the analysis 


eneity of within- 


the attempted 
within classes composed of th 
included in each о) 


each session the probability 
5, this assumption is met sat 
Patterns of Analysis 


It has been noted that the general de. 
sign of the experiment made provision 
for the analysis of the effects upon at- 
tempted leadership of two independent 
variables, task motivation (M) and ex 
pectancy of accomplishment (E). The 
counterbalanced arrangement of the ex- 
periment also provides for the evalu 
(and removal from experimenta] 
of the effects of differences betw 
servers (O). 


ation 
error) 
een ob- 


Session II attempted-leadership oor не 
can be analyzed in more detail than . : 
sion I scores, Of the available 95 n 
of freedom, 29 are assignable to pond 
ences between groups and 72 to n me 
ences among individuals within the gn 
group. The 23 degrees of freedom re 
tween Broups are allocated anong igh 
four experimental treatments (ie. pigh 
motivation and high expectancy; mo- 
motivation and low expectancy, low aoti- 
tivation and high expectancy, wt ree 
vation and low expectancy). ane: ree 
of freedom is allotted to each of ; me 
effects, motivation, expectancy, ane ect- 
interaction of motivation and am 
ancy. Allocations also are made ie 
Sroup differences to provide an eva ob- 
tion of differences between the - cts 
servers who tallied the leadership "m 
and the interaction of the observer m 
the expectancy and motivation poc de- 
The remaining 16 degrees of ae 
pend upon unique differences 2 no 
8toups, ie, uncontrolled variation jon, 
attributable to the effects of er 
CXpectancy, observer differences: ition* 
their interaction. Expectancy condi 
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TABLE 6 


ANALYSIS OF THE VARIANCE OF ATTEMPTED 
LEADERSHIP SCORES FOR SESSION T 


Source of Variation df Variance F 
Between Motivation 

Conditions 1 20.17 6.70? 
pees Observers 1 19:07 3-59 

eraction; Motivation 

Observers 1 ie 

Op Uniqueness 20 meer 

Ithin Groups (Error) 72 

T'otal 5 E 
cum 95 


* H эр 
Significant at the 05 level. 


wer niit ; T 
à = not varied during Session I and the 
analysis is c 4 $ 2 

uysis is correspondingly simplified. 


Analysis of Variance 
ы брі of the Session I scores 
tivation bo evaluation of the task mo- 
айа is Уш iable uncomplicated by vari- 
fion 11 hie tei The analysis ol Ses- 
the eff ар diei an examination ol 
T ects of expectancy, task motiva- 
T. and their interaction. 
"ew 6 presents the results of the 
Scores of = the attempted leadership 
аы € ession I. 
9r Session I the effect of the motiva- 


tion с i 
" conditions 
evel, 


Varie 


is significant at the .05 
Expectancy conditions were not 
not n ашп Session I and therefore do 

pear in this analysis. The F value 
c oe, A Ratociated with group uniqueness 
cance i reach the .o5 level of signifi- 
group, X; we may assume that the 
empted г not differ with respect to at- 
si eadership during the first ses- 


lon, ey Ы А 
Чоп, xcept for the effect of task motiva- 


Ta 
Vari ble 7 presents the analysis of the 
Nance 


Scores for of the attempted leadership 
+ tor Session II. 

tempted В of the variance ele 

3 Tats leadership scores, summarize 

betwe е 7, shows that the differences 
€n observers contributed to the 


variation of the scores for Session 11. We 
also note a significant interaction be- 
tween observers and expectancy. This 
finding contrasts with the failure of the 
observer differences to appear as a signifi- 
cant source of variation during Session I 
(Table 6). 

Both task motivation and expectancy 
are significant sources of variation in at- 
tempted leadership scores for Session 11. 
The interaction between expectancy and 
motivation does not reach significance. 
Apparently, the two experimental treat- 
ments have independent effects upon the 
attempted leadership scores. 

The between-observer variance may be 
attributed to either (a) differences be- 
tween the observers’ standards or (D) dif- 
ferences in the subjects’ behavior associ- 
ated with the time within the day at 
which the experiment was conducted. It 
has been mentioned that one observer 
was employed during the morning run 
(8-10 a.m.), and the other worked only 
in the afternoon (1-3 p.m.). 


TABLE 7 
ANALYSIS OF THE VARIANCE OF ATTEMPTED 
LEADERSHIP SCORES FOR Session II 


= МЕЕ А Vari- 
Source of Variation df nae F 
Between Expectancy 

Conditions т 33.84 тт.87* 
Between Motivation 

Conditions 1 14.26  5.06* 
Between Observers Її 33.84 11.87% 
Interaction; Expectancy 

x Motivation 1 .10 .o4 
Interaction; Expectancy 

x Observers 1 12.76 4.48* 
Interaction; Motivation 

Observers I 1.76 .62 
Interaction; Expectancy 

x Motivation X Ob- 

servers I .02 „от 
Group Uniqueness 16 3.42 1.20 
Within Groups (Error) 72 2.85 

Total 05 


* Significant at the .os level. 
** Significant at the .or level. 
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TABLE 8 


AVERAGE ATTEMPTED Бань SCORES 
ва TED W Two OBSERVERS F 
SOCIATED WITH E 5 F 
ie CONDITIONS OF PECTANCY 


ui Average 
Observer and Condition 


Score 

di ineetance m 

ver А and High Expectancy 3.6; 

[mns B and High Expectancy 4-08 

Observer A and Low Expectancy 1.71 
Observer B and Low Expectancy $5 


63 


We have one bit of evidence which suggests 
that diurnal effects (perhaps related to the sub- 
jects’ alertness in the OHNE) 1 Uie ien 
explanation, Table 8 shows the average A 
tempted leadership scores based on the tabu a- 
tions of the two observers for the two conditions 
of expectancy. The obvious discrepancy in 
Table 8 is the Jow average attempted leadership 
score for Observer A under conditions of low 
expectancy. This observer was employed for the 
8-10 a.m. runs, 

If the subjects who were run in the 
sessions were more alert. than 
run in the afternoon sessions 
perceived. the Objective. condi ions of low 
pectancy more clearly (i.e., realized that it M 
impossible to make profit or point gai 5). 
amination of the Post-Experiment Questionna 
responses to the item, “During the second work 
session, how difficult did you find it to do well" 
reveals that 15 of the 48 group members who 

enced the low "Npectancy condition chose 
the respons ‘Impossible to do well,” 
response was entirely consistent 
jective fac Twelve of the 15 members giving 
this response were from groups run in the morn- 
ing sessions. In other words, 50%, of the 24 sub- 
jects who participated in the experiment in the 
morning perceived the situation correctly as 
compared with only 12.575 in the afternoon, 
The difference between these percentages is 
significant at th 


better 


morning 
those who were 
they may have 


(i.e., their 
with the ob- 


€ 01 level, Tt may be, therefore, 
that the obtained variance between “observers” 
indirectly reflects the effect of time of 


day upon 
icy condi- 


the subjects’ 


responses to the EXpectar 
tions, 


In summary, the results show th 
desired experimenta] were 
established and that the hypothesized 
relationships are reflected in the data, 
Task motivation and expectancy of task 
accomplishment both are significantly re- 
lated to attempts to lead, [i has been 
demonstrated further that these two ү 


at the 
conditions 


ari- 


; E" i 
ables operate independently in the 
effects upon the dependent variable. 


DISCUSSION А 
In general, the results of the cad 
ment support the hypotheses that a 
vided the impetus for the present ien y 
and support the theoretical oir iut 
Which they are related. Individuals “а 
tempt to lead more frequently when и 
rewards for mutual problem solution 
relatively high. They also tend to E 
tempt more leadership acts il e va 
tempted leadership act has a reasona " 
possibility of contributing to pde 
problem solution than if attempts to lea 
cannot accomplish the task. | 7 
Even so, a number of specific ari 
have operated to minimize the eviten: 
effects of task motivation and expec pen 
on the frequency of attempting Ag pe . 
In the two immediately following T 
tions these factors will be discussed Б 
detail. Such ап examination may quel 
tate further efforts to specify more nl 
cisely the functional н Же 
tween task Motivation and expectancy ~ 
those variables determine the frequency 
of attempts to lead, 


Factors Related to Task Motivation 

The design of 
two differentiated 
established for 


ad that 
the experiment stipulated ^ be 
levels of task тераа 
the individual, such that pi less 
to solve the mutual problem would involve the 
dissat faction under one condition Ый, e 
other, The degree to which those indivit tion 
Who work under conditions of high motiva 


: jon 
А ө А > situato! 
feel and show dissatisfaction with the rd that 
а 


m should be greater » Checks 
ers with low motivation. “the К 
SS of the efforts to es ыы јс 
indicated that (a) more sel 
Well on the task as extrem 


'Wo conditions 
regarded doing 


er 
Important under high motivation. than ig 
low motivation and (b) the observers tend wing 
rate the highly motivated subjects as e 
More tension 


and enthusiasm, There s who 
ever, little to Support the view that those 

Worked under the | 
low motivation in an 


‘ions һа! 
motivating nag a 
absolute sense. Desp 


١ 
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the changes made in the task instructions in an 
effort to induce lower motivation, only approxi- 
mately доб, of the subjects indicated that it was 
NOU Very important.” Most subjects found suffi- 
Cent reasons for regarding the task as “relatively” 
or "extremely 7 important. 
l Perhaps more extreme measures could have 
Seen taken to lower task motivation, Yet. there 
ae questions to be asked about whether the 
um NE were made in the present Dusk 
valle ee altered it in more than its incentive 
. Gould the effects attributed to task motiva- 


tion be à д F 
of be related to differences in other properties 
T the task? 


The ло . 
mene differences between the high and low 
ШТП "ү р 3 H 
a vation conditions were introduced. in an 
Attempt уе 


reduce the intrinsic interest. of the 
task, ce the intrinsic intere 


Vo review, the following changes were made: 
a, э? А Я 5 з 
Identification of the task as a " Linkertoy 


Construction 


m test” as compared with a "small 


at Й : 

pufacturing enterprise.” 

aus UB use of "points" ах compared with 
doi al money in the transactions involved. in 

Ping the task. 

& Ap ч . | 
ассо Accumulation of points as the reward for 
В Mpli А Н 
Acta lishment as compared with retention of 

Fi ra wy profits, 
miti Requiring that orders for parts be sub- 
only in triplicate as compared with requiring 
y T order form. 
аз с Requiring that parts be ordered separately 
Yu A: сч э 
for e “pared with permitting orders to be placed 

“kits” of parts, 

The last 


two ang (da ^) were intro- 
duced "— changes (d and с) 


y making 1. the "routine" character of the task 
js aking it repetitive апа boring for the sub- 
first glance, it would appear that these 


буо ch; 
atten, RS might restrict the opportunity fof 
in ‘pled leadership. Observ tions of the group 


acti 3 surmise 
on, however, did not be: surmise. 


пе; is 
0 n of restricting the organizing requirements 
in k through involving the group members 


Urely Y : Pues OR E " 
dert routine individua] activities. these 
in л) of the task provided а sphere of work 
No die attempts to lead were appropriate, 

Merene А : а Сао 
or Crences were apparent in the opportunities 


йөрүн ppo! A 
ask lempting to lead under the high and low 


$ Motivation conditions. 
"acto. Ў 
lors Related to Expectancy 


tap Of the most interesting results of b 
Efect nent was the absence of a significan 
Upon attempts to lead of the interaction 
ds motivation and expectancy. It 
low e not have been surprising if. under the 
Xpectancy condition, the subje working 
prope, iB task motivation had disp ved a dis 
Cad q lonately greater frequency of attempts to 
ne EM those working under low motivdtioh: 
ud have reasoned that the highly moti- 


Ung 


vated subjects spent a disproportionately greater 
effort exploring all possibilities before accepting 
the objective limits of the situation, The facts 
groups that 


are, however, that two of the 
were run under high motivation and low ex- 
pectancy quit work before the end of Session IH. 
The remaining four groups contained at least one 
member who insisted on working until the end 
of the period. 

Under the low e 
for continuing wor 
group members, Such comments as, "We can't 
lose," or "We're supposed to do something,” 
were made frequently by опе or more subjects. 
It seemed incredible to some subjects that. they 
should encounter a situation in which hard work 
could. not accomplish. the task. Much time was 
spent in the early part of the session in checking 
and rechecking the available information. In 
fa many (more than one-third) of the acts 
of attempted leadership were made before all the 
members of the specific groups had realized that 
nothing was to be gi ined by trying. 

Individuals could—and. frequently did—make 
errors of reasoning that resulted in their drawing 
erroneous. conclusions about the possibility of 
accomplishment, If it had been feasible to eli- 
minate from the analysis all attempted leader- 
ship acts made when their authors believed that 
a profit was possible (even though it was not). 
the effect of expectancy on attempted leadership 
would have been even more striking. 

The subjects began the experiment (Session 1) 
under expectancy conditions that proved to be 
similar to the high expectancy conditions of 
Session Il, The fact that larger rewards were 
ble during Session H did not appear to 


xpectancy condition, reasons 
were often discussed by the 


р‹ SS i | 
have a noticeable effect. upon most subjects. 
Only rarely did a group member comment upon 


the increased attractiven of the profit situa- 
tion, The number of leadership acts attempted 
during Session IT under conditions of high ex- 
pectancy was not significantly larger than the 
number attempted during Session I. 

There remain many problems of specifying 
more precisely the relationships between the 
frequency of attempts to lead and (a) the ob- 
jective characteristics of a situation, (b) the per- 
ceptions of these characteristics by members of 
the groups. (c) the related expectancies of prob- 
lem solutions, and (d) the degree of certainty 
sociated with these expectancies. The present 
results are limited in showing only that major 
differences in the objective situation do affect the 
frequency of attempted leadership acts. 


Pre-Established Structures-in-Interactions 


During Session I many groups were successful 
in establishing patterns of organization that per- 
sisted with varying consequences into Session 11. 
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It was possible for the group members to up 
set ways to perform the task because 

ue 1 unchanged in the second session except 
[egi дш of a new profit or point 
schedule. The main activities of ud con- 
tinued to be to secure parts from the experi- 

nter by using the order forms, to assemble 
the same kinds of toys, and to return finished 
айй to the experimenter for credit, If, tor 
example, the members of a group had worked 
out a seemingly efficient division of labor for the 
production of “tops” during Session I, there was 
a tendency for them to continue with the same 
organization and to build tops during Ses- 
sion II. | 

Such a consistency in the behavior of a group 
is referred to in the present theory (3) as a 
“pre-established structure-in-interaction.” It is 
also suggested in the theory that these structures 
may replace leadership behavior, but at a cost 
of flexibility in the solution of group problems 
(3, рр. 17, 32). Both circumstances were noted 
in informal observations of the experimental 
groups. Some groups continued to build the 
same toy in the second Session tl 
built in the first, суеп after the: 


during the second Session obscuri 
ancy variable by diverting the attention of the 
group members from other more favorable alter- 
natives, and (b) the number of leadership acts 
that might otherwise have been attempted during 
the second session Was reduced by a single de- 
cision simply to adhere to the prior arrange- 
ments, 

These observations concerning the function 
of pre-established str i "interaction, al- 
though consistent With the theory, serve only to 
Suggest additional controls that might have been 
employed іп the experiment, Single sup. 
stitution of different mod 


in expectancy, 
design might readi} 
produce more conclusive results. However, What 
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is perhaps the next step in the ШУМЕШ af 
these problems is to begin to specify tae Я И 
what properties of tasks аге most гербиш E pr 
the observed effects upon attempted leaders 1 
In this way it may be possible to predict par 
vidual attempts to lead, from knowledge ag 
motivational and expectancy characteristics 
various situations, 


FINAL SUMMARY AND CONCLUSIONS 
Summary of the Experiment 

This experiment, the third of a вее 
concerned with attempted leadership, 
Was designed to test hypotheses oe 
ing the effects of task motivation and ex 
pectancy of accomplishment. | idi 

During the first 25-minute session, е 
experimental groups were run unc i 
either high or low task motivation am 
constant expe Чапсу conditions. emen 
the second session, four RPE 
conditions existed, namely, (a) high 
motivation and high expectancy, (b) hig 
task motivation and low expectancy, (0) 
low task motivation and high apar 
ancy, and (d) low task motivation E 
low expectancy. The data obtained due 
ing the first session provided for an a 
complicated test of the effects of es 
motivation оп the frequency of qi inn 
leadership, The effects of both variab E 
and of their interaction could be studiec 
by analysis of the data collected during 
the second Session, e 

The task used in the experiment ee 
а modified form of the Manufacturing 
Problem, It involves consigan e 
models from Tinkertoy parts pape 
from the experimenter in exchange a 
points or Money with the object of dins 
ing maximum gains from the manu E 
ture and sale of toys to the aa. 

Wenty-four Broups of four men нун 

were run during the experiment. ачаан 
еасһ of the four experimental conditio 1 
Six groups Were assigned in a “ered 
balanced order to morning and alte 


— — Ӯ, 


0 ——— M — ^Qeadg— аш 


mant 
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те м > that interobserver differ- 
ба] texts uld be removed from the statisti- 
tests of the eflects of the independent 
variables, 
te n variable was the fre- 
tally whe ашайма leadership acts. A 
kach rie mada by a trained observer 
lead a group member attempted to 
analysis ee As the first step in the 
ormed e frequency data were trans- 
analysis н иаша appropriate for the 
Ep. of variance. 
dew E of the main analy 
Potheses | ape cach of the three hy- 
menta] dtes each of the two experi 
tion es 271905 the effect of task motiva- 
Pectaney ааа at the .05 level. Ex- 
Sion eL ee during the second 5е5- 
The "Јас significant at the .o1 level. 
Motivation war of expectancy and task 
“he ae os ү 
Cussedl я “ef = ings have ›ееп Ше 
Vations th: reference to informal obser- 
5 that suggest ways of improving 
ding investigation of the varia- 
ask motivation is difficult to estab- 
Signed is ЗИ level, and procedures de- 
have баа ыу this variable may 
Properties a effects upon other 
седеа in ha the task. F urther work is 
Ë the carti der to clarify the operation 
tion, dame involved in task motiva- 
ееп E ects of expectancy may have 
ally obscured by events inter- 


and exten 
les, T 
lish at 


vening between the objectively defined 
situation and the individual's interpre- 
tation of it. One special type of such 
intervening event is the early establish- 
ment of structures-in-interaction that 
may influence both the requirements for 
attempting leadership and the kinds of 
decisions that are made in the face of 
the facts of the situation. 


Conclusions 

In general, the conclusions of the ex- 
periment are as follows: 

1. Individuals will attempt to lead 
more frequently if the rewards for task 
solution are high rather than low. 

о, If individuals expect acts of leader- 
ship to result in problem solution, they 
will attempt to lead more frequently 
than if they see no relationship between 
leading and solving the problem. 

3. In this experiment, task motivation 
and expectancy did not interact signifi- 


cantly in their effects upon attempted 


leadership. 
4. In the design of further experi- 
g these variables, it is rec- 


ments employin 
attention be 


ommended that careful 
1 to both the potential effects upon 


task characteristics of procedures 
ary task motivation and the 
may intervene between 
esented and the situa- 


giver 
other 
utilized to v 
processes that 
the situation as pri 
tion as perceived. 
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The Pensacola Z Survey: A Study in the Measurement of 
Authoritarian Tendency 


MARSHALL D. JONES! 
U. S. Naval School of Aviation Medicine 
Pensacola, Florida 


3 ies MHEORY of authoritarianism is in- 
a of political affiliation. 
authori, б, Ше principal measure of 
is Rode. the California F Scale, 
tentia] S Xm measure of fascist po 
Scale i; M Thus, in principle, the Е 
Of varia, BI inner: of at least two sorts 
Politica] с authoritarianism and socio- 
study 2 тешу, Clearly, for the 
measure nope ripis: generally, a 
desired : e specific politically is to be 
. . The purpose of this monograph 


IS th 5 : 
ü € construction of such an instru- 
nent. 


political particularity of the F Scale 
I БО the fact that its items are all ex- 
1€ items 3 specifically “Fascist” point of view. 
in бы а attitudinal in form and sociopoliti- 
Only mat ntent, These, of course, are not the 
Onnajre 1415 from which a personality ques- 
direc а May be constructed. In fact, the in- 
meas, PProach through social attitudes to the 
Ment of personal qualities is rather un- 
Much more common are items which 
rectly with the subject: his habits, tem- 
nt and reactions, Importantly, the usual 
Broup; es not concern contemporary social 
Particular Policies, If we are to avoid political 
Чааны 7 їп the measurement of authori- 
irec ae we might well rely exclusively upon 
Polit; | terials as devoid as possible of socio- 
. Teference. The construction effort re- 


Сопу, this study has been dominated by this 
eration, 


usual 


The y 
© results of this effort are two test 


Opin; 
Pa е mions Or conclusions contained in this 
һе AN those of the author. They are not to 
Or the | Wed as necessarily reflecting the view 
endorsement of the Navy Department. 


instruments: the Pensacola Z Scale and 
the Pensacola Z Survey. The first of these 
is a single-score test constructed against 
the F Scale but composed of materials 
as "asocial" as possible. Since the word 
"authoritarianism" has become insepa- 
rably associated with the F Scale, we will 
use the word “heteronomy’? and its af- 
fines when speaking of the Z Scale. The 
word “authoritarian” and its affines will 
be reserved for the F Scale. The Z Survey 
is a five-score instrument. One of these 
scales is the Pensacola Z Scale itself. The 
remaining four, labeled Dependency, 
Rigidity, Anxiety, and Hostility, are ex- 
pansions of the principal cluster-compo- 
nents of the Z Scale. 


SETTING 

The Pensacola Z Scale is essentially a 
measure built from ordinary “personal- 
ity" items against an attitudinal measure 
of fascist potential. The Z Scale is not, 
however, the first measure that fits this 
description. In 1951 Gough (3) reported 
the construction of his Pr Scale. This in- 
strument was built from items of the 
Minnesota Multiphasic Personality In- 
ventory (5) against the A-S (Anti-Semi- 
tism Scale of the California Group (1). 


2The term “heteronomy” has been used by 
Riesman (16) in very much the same sense as 
in this study. Although the present usage implies 
specific tests, Riesman’s understanding of the 
term can be taken as the conceptual intent of 
the Z measures. 27 


Bur^^. Edi. 
» 
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The Pr Scale was not completely ae 
veloped; no retest reliabilities, loy = 
ample, were reported, NENEMUIGING, the 
Pr Scale could have been our starting 
point. Instead, we chose to start from 
scratch and build the Z Scale. The rea- 
son for this choice lay in the content of 
the Pr Scale. No less than eight of the 
items of the Pr Scale are virtually iden- 
tical to one or another F-Scale item. At- 
titudinal material of a nature specific to 
fascist potential was not deleted. This, 
of course, is no criticism of Gough. His 
intent was not to create a scale free from 
social content. For our purposes, how- 
ever, the inclusion of these items would 
have prejudiced our entire undertaking. 


All subjects involved in this work were Naval 
Aviation Cadets, The cadets are procured from 
two sources: civilian life and the fleet. Civilian- 
procured cadets comprise approximately 80% 
of the total. All cadets procured from civilian 
life must have had at least two years of college. 
Fleet-procured cadets must have graduated from 
high school, All subjects were tested within a 
week after reporting for duty, when the cadets 
from civilian life were still in civilian clothes and 
the fleet-procured cadets still in their enlisted 
uniforms. All cadets, whether civilian or flect- 
procured, must pass the sclection battery of the 
Naval Air Training Program, be single, and 

between 18 and 25 years of age. All sections of 

the country are represented in any sample of 
appreciable size. 

The properties of the F 
population are, of course, a matter of particular 
concern. In the first place, the original 30-item 
scale was slightly modified, One item, "It is best 
10 use some pre-war authorities in Germany to 
keep order and preserve peace,” was dropped 
as no longer topical. The form used in this study 
had, therefore, 20 items, i 

Means and standard deviations о 
are generally reported in item units 
the mean or standard deviation 
for all go or all 29 items, as the Case may be. The 
grand mean is then divided by the number 
of items involved, In these units the Е Scale in 
the cadet population has a mean approximately 
equal to 3.90 and a standard deviation ap- 
proximately equal to .75. These parameters ap- 
plv approximately to all of the samples studied 
in this work. Neither the mean nor the variance 
of the F Scale in the cadet Population departs 


Scale in the cadet 


f the F Scale 
. Specifically, 
is calculated 


» 
+ 


А мей duo The du 
radically from the values reported in ses v 
thoritarian. Personality. In a typical (gon : 
311 cadets the Е Scale was distributed s E 
к H я kur 
negative skew, g, = .31. and a slighi platyku 

tosis 


= —.o8. " 

‘The homogeneity of the F Scale in shines s 
of 291 cadets was found to be .81, using the 
Hoyt analysis of variance technique (8). This co- 
efficient is somewhat lower than the value re- 
ported in The Authoritarian Personality. m 

The 2j-hour, test-retest reliability of ee 
Scale in a sample of 8o cadets was found to te 
+.90 (6). This result corresponds closely to 1 р 
results obtained in the original study, es 
d-week, test-retest coefficient is ay o 
lower, +.76. The correspondence of this үс: 
to the original work cannot be ura 
there was no comparable coeflicient quoter M 
the California study. The drop is not, however, 
as dramatic as it may seem, In the cadet eni 
tion, almost all personality schedules with rete ó 
coefficients in the high .80's or low .go's a a 
hour retest drop to the mid 70's at geweek retest. 


PREPARATION 

Before the Pensacola Z Seale could he 
constructed, a preliminary pool of iow 
had to be created. In writing these items, 
we had the choice of taking items more 
or less at random or of attempting some 
sort of organization. In the latter event 
we could have resorted either to a theo- 
retical or 


fo ап empirical basis. ва 
were used. The studies which follow 
were conducted so that we might eee 
some idea as to what sorts of items аб 
be expected to relate to the E Bw 
These relations might then provide xà 
empirical basis for the writing [ча 
Altogether there were four studies. ; 

each of these, a st A 


andard personality 
ven tory was 


administered in conjunction 
with the F Scale. The four inventoric* 
were: the Taylor Manifest Дый 
Scale (17), the Wesley Manifest Rigid! | 
Scale (20), the Guilford-Zimmerma” 
Temperament Survey (4), and the 1 i! 

stone ‘Temperament Schedule (18). * 


anc 

four measures had been developed 

Е: нн " istics, Bt 
"For а precise definition of the statis 

and g,, 


see McNemar (15; pe. 29): 


oo — 
—————- 

—— 
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THE РЕ 


CORRELATIONS OF THE F SCALE WITH THE 
SURVEY AND WITH THE SEVEN SC: 


SACOLA Z 


SURVE 


TABLE 1 
=x SCALES OF THE GUILFORD-ZIMMERMAN T 


PERAMENT 
MPERAMENT SCHEDULE 


lHURSTONE 


M 


бае |G R А 0 F 

Scales from the GZTS | ло p SPF .o8 .12 .22 —.24 —.27 —.05 وفك‎ 
. wv т P E 5 р 

Scales from the TTS | LA j a3 i ‚эз E SAMO = 23 0408 


® Each capitalized letter denotes the scale of the GZTS 


Involved. 


were intended for use in “normal” popu- 
lations, The principal consideration that 
Prompted this restriction was that the Z 
Scale was designed for use in normal 
Populations. 


reat samples of 166 and 245 cadets, re- 
shang e 1 ak correlations (14) of +23 and 
. € obtained between the F Scale and the 
Mi, Manifest Anxiety Scale. The first co- 
the sec xy ы ү, beyond the .o2 level and 
Study E beyond the ‘oi level. In. the second 
tween tals SOREIRHOR of 4.19 was found be- 
Rigidity Seal Scale and the Wesley Manifest 
Tesult scale; ina sample of 221 cadets. This 
the ang ee beyond the 001 level. In 

emperam | study the Guilford-Zimmerman 
Sample sees Survey was administered to a 
т 28 cadets together with the calc. 


le te 
tiva, беп Seales of the GZTS are: General Ac- 
|н Restraint, Ascendancy, Sociability, Ob- 
ly, Friendliness, Personal Relations, 


ho 
Ughtfulness, and Masculinity. Of these, four 


ad E ‹ 
signings ations with the F Scale which were 
four icant beyond the .o1 level. Two of the 


ay ee Stability and Objectivity, are 
ation". oaded with anxiety items, an interpre- 
6 which is supported by the correlation of 
Wo sich obtains between them, The other 
also nce Friendliness and Personal Relations, 
tion hee, a heavy common factor, the corrcla- 
hostility Seen them being 4.59. In this case 
correla items appear to be the basis of the 
Tem Y OTi In the fourth study the Thurstone 
Perament Schedule was paired with the F 

of i а sample of 304 cadets. The seven scales 
omin TTS are: Active, Vigorous, Impulsive, 
e eae Emotionally Stable, Sociable, and 
ive. Of the seven scales, four had correla- 


t; 


4 
Fi 
ide a fuller account of the preparatory 
a i sce the original reports (11, 12, 19). 

ia Significa in this study are 
two-tailed. ance levels quoted y 


st 


or of the TTS which begins with the letter 


tions with the F Scale which were significant 
beyond the .o! level. The first of these, Emo- 
tionally Stable, is, like Emotional Stability in 
the GZTS, essentially an anxiety scale. Its re- 
lationship with the F Scale constitutes, therefore, 
the fourth positive relationship between anxiety 
and F-authoritarianism, The second scale to re- 
late significantly to the F Scale was Sociable. 
This scale, like its counterpart in the GZTS, 
Sociability, is largely a measure of social ex- 
trav on. Sociability, however, did not relate 
significantly to the Е Scale. The third signifi- 
cantly related scale of the TTS was Impulsive. 
The Restraint Scale of the GZTS corresponds, 
inversely, to the Impulsive Scale of the TTS. 
In the third study Restraint was related at the 
оз level to the F Scale, and in the consonant 


direction. 


All in all, the four studies yield a re- 
arkably stable portrait of the F-Scale 


m 
The principal features of 


authoritarian. 
this portrait are anxiety, hostility, rigid- 
ity, social extraversion, and impulsive- 
ness. 

CONSTRUCTION 


Of the five traits established by the 
preparatory studies as authoritarian only 
three, anxiety, hostility, and rigidity, 
were deliberately utilized in the writing 
of items. The principal reason for ex- 
cluding social extraversion and impul- 
siveness was that their relationships to 
the Е Scale were both weaker and less 
consistent than those of the three in- 
cluded traits. A second consideration was 
that in the author’s judgment the two 
excluded traits seemed much less cen- 
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tral than the three included traits to the 
authoritarian syndrome. This second 
“consideration” was, of course, entirely 
subjective. The preparatory зше; 
then, resulted in three core traits toward 
which the items of the Z Scale could be 
oriented: anxiety, hostility, and rigidity. 
To these three a fourth, dependency, was 
added without empirical justification. 
Logically, dependency is the essence of 
the heteronomous syndrome. Unfortu- 
nately, no adequate, objective scale of 
dependency exists. Dependency, there- 
fore, was added on purely а priori 
grounds. In sum, then, the items of the 
Z Scale were oriented tow 


ard four core 
traits: anxiety, hostility, rigidity, and 
dependency. In writing th 


€ items, every 
effort we 


, of course, made to exclude 
sociopolitical refere: 
ble. 

Altogether 200 


nee insofar as possi- 


items were created.’ 
АП were written in forced-choice form, 
The principal consideration which led 
to adoption of the forced-choice form 
was the relative unfakability ОГ forced- 
choice items. The F ale itself is subject 
to faking (s). The forced-choice 
seemed, on available evidence, to offer 
good promise of protection against fak- 
ing. To what extent these 
realized is a matter which w 
up later. Of the 209 items, 
iented toward anxiety, 
tility, 50 toward rigidity, and 5o toward 
dependency. The “orientation” of the 
items was, of course, largely subjective 
since the author's judgment 
crucially between the results 


form 


hopes were 
ill be taken 
50 were Or- 
50 toward hos- 


intervened 
of the pre- 
° The items of the Taylor 


and Wesley Scales, 
the Guilford Zimmerman 


Temperament Survey, 
the Thurstone ‘Temperament Schedule, and. in 
particular. a forced-choice test of leadership (9) 
which was developed by the Personne} Research 
Branch of the Adjutant General's Office 
West Point cadets were al] utilized in 
of items for the Pensacola 7. Scale, 


with 
the writing 


paratory studies and the writing ol ilig 
items. As will be seen, however, this 
subjective orientation ol 
in the main at least, on c 
results, turns out to be 8 


items based, 
rlier objective 
o effective in 
estimating the direction, if any, in which 
an item will relate to the F Scale. Го 
What extent the classification of items 


i i © justified is 
Into four categories can be justific 

i D { iz „алеп up 
again a question which will be taken u] 
later, 


The 200 items were prepared: together en 
instructions. and administered to gob cadets {ог 
most immediately after thev had reported s 
duty at the Naval Air Station, The F Scale wi 
administered. one day later. ‚сайа 

On the basis of their E scores the 306 pos 
were broken down into three ШОШ E ils 
who scored in the lower "soon the Ё Mns 
those who scored in the middle ШО era 
Who scored in the upper ‚© Rank core 
Cocllicients were then obtained. for cast 
against the trichotomized F Scale. 

In evaluating апу given item, у: „те 
criterion was imposed: (a) the item ha nc: 
late to the F Scale in the theoretical ee bes 
pated direction, (bj less than куе, of the and 
had to answer the item in the same МТ: r 
(c) the significance of the item Ca bet- 
Scale had to re 
ter). ОГ the 
beyond the 


ach acceptable levels Gar nificant 
200 items, 78 were sign те 
20 level, OF these, по did net a 
late to the F Scale in the theoretically, „= а 
pated direction; to an additional two i^ the 
Majority greater than 8 responded ише: 
same way, ‘The remaining 66 items AP шүк 0 
the Scale. In the appendix the GO It 
the Z Scale аге reproduced, alysis are 
In Table 2 the results of the item analy the л 
set forth, In Column ı1 the items of 


bers 
ў } ‘The num 
Scale are identified by number, The nn 
correspond to 1 


, ) 
hose in the appendix. m ропе 
2 the Proportion of subjects who gave sp of sig 
SA” 18 noted; and in Column 3 the leve ло. 
nificance to Which the item attained. ( 
5 ог wi): In the appendix the eae 
and the in 


20. 


иеп! 
authoritarian response of 
are indicated, 

In 
were 


MET 
Cossvalidating the Z Scale the ed ас 
Prepared as in the Appendix on the Т 
ministered to 311 cadets who had [е scale 
Seale one day ea lier. In. scoring the vitari 
unit weight was assigned to cach айт е 
response, Thus, the maximum score pre 

66 and the minimum o, 


o 


à 


PROPORTION OF 


AND SIGNIFICAN 


Item 


to 
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о жм 


THE PENSACOLA Z SURVEY 5 


TABLE: = 
SUBJE 


1. 


у 


ст 


S FOR THE 661 


RESPONDING 


OF THE PENSACOLA Z SCALE 


Significance 


TABLE 2 (continued) 


Item Proportion “А” 


Significance 
.20 
02 .20 .20 
63, + -33 Jos 
64 оо +20 
05 71 ло 
66 ED 220) 


The Z Scale upon crossvalidation 
correlated. +.43 with the F Scale. No 
attempt was made further to refine the 
Z Seale. One reason for not refining the 
Z Scale further was that the crossvalida- 
tion coefficient of +.43 seemed quite 
adequate as it stood. A second reason 
rested on the reduction in the number 
of items which would in all probability 
have resulted from further refinement. 
The Pensacola Z Scale was designed to 
serve (a) as a terminal measure of per- 
sonal autonomy and (b) as a seed scale 
from which the Pensacola Z Survey 
would be built. The Z Survey was to 
be constructed about the item clusters, 
presuming their existence, in the Z Scale. 
This approach demanded a fairly sub- 
stantial number of items in each cluster 
about which to build the independent 
scales of the Z Survey. Indeed, simply 
to determine whether or not there were 
item clusters in the Z Scale demanded 
that there be more than three or four 
items in a cluster. Further refinement of 
the Z Scale might, through a reduction 
of items, have jeopardized both the de- 
termination and the expansion of item 
clusters within the Z Scale. For these 
reasons, no effort was made to improve 
the crossvalidation coefficient of 4-48: 


То determine the test-retest reliability 
coefficients of the Z Scale, two distinct 
samples were used. The first sample, of 
187 subjects, retook the Z Scale after 24 
hours; the second sample, of 199 sub- 
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jects, retook the test after four weeks. In 
the 24-hour sample the test-retest co- 
efficient was +.87; in the four-week 
sample the value was +.74. А 

The notion of "fakability" is com- 
pounded of two distinct meanings. The 
first of these is that a nonfakable test 
is one such that the average subject 
cannot improve his score by faking. The 
second meaning is that a nonfakable test 
is one on which the average subject can- 
not appreciably alter his score by faking. 
The fakability of the Pensacola Z Scale 
was examined in both senses. 

To determine fakability in the first 
sense the test was administered to two 
entirely distinct groups. The first group 
received the customary instructions. The 
second group was instructed to "beat the 
test,” to put down what they thought 
was the "best" answer, regardless of its 
truth when applied to them. To deter- 
mine fakability in the second sense a 
single group was used which took the 
test first under the usual 
and then again under 
fake. The correl 


instructions 
instructions to 
ation between the two 
administrations was adopted as a meas- 
ure of "alteration." 

In examining fakability in the first 
sense the Z Scale was administered to 
196 subjects under a set to fake. The 
crossvalidation sample of 311 cadets was 
used as the second group. In the cross. 
validation sample the 7 Scale had a 
mean equal to 35.31 and a standard de- 
viation equal to 6.85. In the “fake” 
ple the Z Scale had a mean equal to 
36.18 and a standard deviation equal to 
5.69. The variances in the two groups 
were unequal, significance reaching be- 
yond the .o5 level. Applying the Behrens. 
Fisher test (10) for unequal Variances to 
the difference between means, We obtain 


a value of d = 1.55 and а value of 
0 = 44^. This result falls considerably 


sam- 


short of the .o5 level of significance. 
From these results we may fairly assume 
that the Z Scale is not fakable in the 
first sense. The significant difference 1n 
variances, however, suggests that some 
items are being faked. Since the mean 
does not change, the cadets must be mE 
ing some items against, and other A: 
with, the key. The faking, however, © 
even a few of the items would be ex 
pected to produce considerable altera- 
tion in the subject's score. -— 
To determine fakability in the secon 
sense the Z Scale was administered tO 
220 cadets with the usual instructions. 
One day later, they took the test under 
instructions to fake. The correlation be 
tween the two administrations was + -5?: 
Under a set to fake, therefore, а sub- 
ject’s score on the Z Scale is subject = 
considerable alteration. Nevertheless, 
there is still considerable communality 
between the test under the two sets. 
The second meaning of “fakability” as a 
tion gives rise to an associated problem: с 2j 16 
extent is the alteration of one’s score тее ei 
intelligence? su 


ý the 
H b^ 6, yen А 
in examining alteration. was also giv 


2 е more iN 
L Scale under the two sets. If the pes the 
telligent subjects can "fake better” th à the 


Sven normally represents the legii 
vantage in autonomy of the intelligent. ^^, then 
dition, the intelligent can “fake ШЕН нё 
to this basal advantage there should 2¢ tative 
the intelligent an increment in Z ptem j ТО 
of their more effective efforts at dishone* P yi. 
test, therefore, the hypothesis that er he 
Sent can “fake better” we need only сотр? scale 
correlations between the ACE and the 
under the two sets, ales: the 
The ACE actually consists of two SC gnd 2 
first is а measure of verbal and the = y rbal 
measure of numerical ability. Since the 
aptitude might be particularly germane 


سے 
Se‏ 
———————— 
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problem, two measures from the ACE were used: 
the verbal score and the total score, which is the 
algebraic sum of the two included scores. The 
correlation between the total score on the ACE 
and the Z Scale as customarily administered was 
—.22; the correlation between the total score 
and the Z Scale administered under a set to fake 
Was —.23. No entirely satisfactory means exists 
for testing the significance of a difference be- 
tween correlation coefficients when the cocfi- 
cients are based on the same subjects. The pro- 
cedure developed by Hotelling (7) is. however, 
the best available. This procedure results in a 
Statistic distributed as the F-ratio with 1 and 
(N-3) degrees of freedom, where N is the num- 
ber of subjects involved, In addition to the two 
correlations involving the ACE, the сог 
between the Z Scale under the two 
Necessary in order to calculate Hotelling's statis- 
tic, The value of this coefficient was, as already 
Teported, +.52. The value of F for the difference 
between the two coefficients involving the ACE 
does not approach any acceptable level of signifi- 
cance. The verbal score correlated —.20 and —.26 
With the Z Scale, normal and * c" respecti 
This difference by Hotelling’s method yields а 
result Which, like the first, falls far short of 
Significance, From these results we may fairly 
m that the ability to fake on the Z Scale is 
ш 2 ated to intelligence. Thus, while the at- 
DL MM may disrupt a subject's score, he 
Rant expect to do better, even if he is intelli- 


In Table g the distribution parameters 
id Z Scale in a sample of 766 cadets 
MIL The negative skew is sig- 
Pare nt at the .o1 level; the tendency to 
l atykurtosis does not reach significance 
i :05 level. The reader will remem- 
that the F Scale also had a negative 
skew and a slight tendency toward platy- 
x The distribution of the Z Scale 
ec therefore, be a fairly immediate 
eur е of its construction. In gen- 

» the tendency to skew increases with 
m m Samples of high heteronomy 
sam, сум skewed than less heteronomous 

ls ce All in all, the distribution of 
töm Scale does not depart radically 
a a The use of normal sta- 
ee Would seem justified except with 
high : of borderline significance and 
eteronomous potential. 


TABLE 3 


DISTRIBUTION PARAMETERS OF THE 
PENSACOLA Z S E 


Parameter 
N 766 
2 35.51 
оз 6.33 
n —.49 
£2 = چو‎ 


The homogeneity of the Z Scale in the 
crossvalidation sample of 311 cadets was 
+.72, This value was obtained using the 
Hoyt analysis of variance procedure. 
The Z Scale is at least moderately hetero- 
geneous, This result is not, of course, 
entirely unexpected. In writing the 
items of the Z Scale, four kinds of items, 
distinct at least by intention, were cre- 
ated. To the extent that this intent was 
realized heterogeneity was to be ex- 
pected. The existence of heterogeneity 
does not, by itself, confirm the fourfold 
classification of Z-Scale items. More exact 
confirmation can hinge only on a precise 
analysis of the item intercorrelations. 

The original pool of items in the Pen- 
sacola Z Scale consisted of зоо items 
written in forced-choice form. Of the 
200 items, 50 were intended to be de- 
pendency, 50 rigidity, 50 anxiety, and 50 
hostility items. Of the surviving 66 items 
of the Z Scale 19 had been originally ori- 
ented toward dependency, 20 toward 
rigidity, 15 toward anxiety, and 12 
toward hostility. If the intent of the au- 
thor in writing the items of the Pensa- 
cola Z Scale were realized in fact, then 
the item correlations within each of the 
four groups of items should be higher 
than the item correlations between the 
groups. In other words, the dependency 
items should cluster together; as should 
the rigidity, anxiety, and hostility items. 

To examine this question the Z Scale 
was administered to 409 Naval Aviation 
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TABLE 4 | 
E ;LATIONS BETWEEN AND WITHIN 
AVERAGE CORRELATIONS 3 à Б 
ain THE Four ORIGINAL GROUPS OF THE 
PENSACOLA Z SCALE 


213$ 065 038 
р ЕА o82  —.00; 
А EN — 140 


| 
| 
| 


Note.—The figures on the principal diagonal 


represent the within group averages. 


Cadets. Less than half of these subjects 
had been included in earlier samples. 
‘Tetrachoric correlations within every 
possible pair (2,145 altogether) of the 
66 items of the Z Scale were then ob- 
tained. Since every item-response was 
scored as either autonomous or heteron- 
omous, the signs of the correlations have 
a uniform meaning. A positive correla- 
tion means that the persons who gave 
the autonomous responses to the two 
items tended to be the same. A 


negative 
correlation me; 


ins that the persons who 
gave the autonomous response to one 
of the two items tended to give 
heteronomous response to the other, 

The average item correlations within 
and between Broups are set forth in 
Table 4. As may be inferred from Table 
4, the average between-group correlation 
is +.030. The average within-group 
correlation is +.109. Even the correla- 
tions of the weakest group, hostility, 
average well over twice the size of the 
between-group mean. The strongest, 
anxiety, averages five times the between- 
group mean. In brief, then, there is good 
evidence that the four core traits are, 


in fact, present in the composition of the 
Z Scale. 


the 


EXPANSION 


The underlying logic of the p 


ensacola 
Z Scale was to organize into 


a single in- 


strument those sources of variance in the 
F Scale which could be incorporated by 
orthodox asocial materials. In a similar 
fashion the Z Scale itself may be frac- 
tionated into component parts. The m- 
ternal constitution of the Z Scale wag 
found to contain four distinct clusters of 
items. These four clusters are distin- 
guished from one another by their el 
tent. They are also distinct statistically. 
In the 7 Survey the four core clusters E 
the Z Seale will be expanded into inde- 
pendently scored measures. 

In constructing the Z Scale the 
of the different sources of variance in Bis 
F Scale was a matter of concern in ио, 
Of particular significance was the m. 
bility that asocial materials might ао 
stract from the F Scale its heteronomous 
heart, its basic authoritarian d 
To know that a person was а hig pa 
scorer was not enough, We had also m 
know whether the result was mediate’ 
by the social content of the F Scale ш 
not. In a similar way, to know that & per 
son is heteronomous is not enough. is 
manner in which a person arrives ide 
high-Z score may well be as significant : 
the fact that he does. 

Before turning lo the ae’ 
process a description of its end rest! 


ature 


tion 


May serve to orient the reader. 11 vow 
Survey has five scales, ‘The first and ШР 
tral scale js Heteronomy. ‘The ir 
omy Scale of the Z Survey is nothing | 66 
than the original Z Scale. The wre 
items of the Survey are the items pen 
Z Scale and they е 


арреаг їп the 
order, 


The remaining four scales кз 
Dependency, Rigidity, Anxiety. and de- 
tility. These, of course, are the cwn 
rived by expansion of the t hr 
ments. In a logical sense they are c 
eral to the centra] scale, Heterono gs 
Each of the four peripheral scales 


e 
——, 


THE 


Jo items, “Phe total Survey has 197 and 
not 


j; items, as might at first be sup- 
posed. There is item overlap between the 
peripheral scales and Heteronomy.. To 
be precise there are 20 items of Heteron- 
omy which occur in one or another ol 
the peripheral scales. Needless to sav. 
there is no item overlap. between the 
peripheral scales themselves. In deserib- 
Ing the process whereby the Z Survey wis 
created the construction of the periph- 
cral will be taken up first. We 
Will then consider. the question. as to 
What effect, if any, the incorporation ol 
the Z Scale into the Z Survey as Heteron- 
SU has had upon the properties ol the 
original scale, 


scales 


р The Z Survey was constructed from a pool of 
уд OF the goo items there were 75 in cach 
ete (dependency, rigidity, anxiety, and 
origin He Т he first 66 items were those of the 
Posi Z. Scale, Of the remaining 234 items ap- 
or it ee one-third were taken over intact 
МА Nery little modification from. the fist 
йв ey item-pool of the 7 Scale; the remain- 
hinge is newly written, Naturally, in writing 
the 7 ane those items which clustered best in 
| Z Scale were used as models. 
atten Serio of the four peripheral scales 
in six Sie had been created was carried out 
Administ si lhe first of these steps 
Tic ices the second. item-pool to. 310 cadets. 
sacola 7 бы ا‎ were the same as for the Pen- 
down "m Seale. The 300 items were then broken 
each of A the four groups of 75 items each, For 
Choric 5 Majo groups à complete matrix of tetra- 
Tha ب‎ ions Was obtained. In every E 
lion ems were scored as cither autonomous. or 
exui ie UR. ‘Thus, in the anxiety. set, for 
ticas €. a positive correlation between two it 
Spot that the cadets who gave the anxious re- 
Nisin to опе of the two items tended to give the 
corre MS response to the other as well. 4 nega 
ünxio, Ung means that the cadets who gave the 
te ast ae response to one of the two items tended 
Ones the nonanxious response to the other. 
Y ле intercorrelations within each group of 
tion (Us had been obtained, we were ina posi- 
this bebes and pick out the best items. From 
Mems ae € ай me P E gup RA. 
was made cated as entirely ишпе, хо i 
Broups, Г] to obtain the correlations. between 
Was cas he final Dependency Scale, for example, 
"structed exclusively from those items 


was to 
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TABI 
CONSTRUCTION DATA FOR THE 
PENSACOLA Z SURVEY 


Them D R A H 

Total 75 
(ау. interitem 7) .108 .130 .182 .099 
Items eliminated 9 3 1 7 
Working pool 
(ау. interitem 7) | .697 «120 .т7; «одї 
Final 40 | 

у. interitem 7) | 107 200 

1, homogeneity | .a1 -04 
Actual homogeneity .86 оо 


which had been written for dependency. Thus, 
there was no possibility that ап item. written 
originally for rigidity, say, would end up in the 
Dependency Seale. This 1 striction clearly. pre- 
vented the full exploitation of the information 
contained in the complete goo-item matrix, The 
reasons for not obtaining the complete matrix 
were almost purely practical, goo-item matrix 
involves the calculation of almost five times as 
many coellicients as does the calculation of four 
туйеш matric ‘The expected. yield, on the 
other hand, did not appear to justify the increase 
in labor, The second step in the construction of 
the Z Survey was to calculate the average inter- 
item correlation in cach set of 2,775 coefficients. 
In Table 5 these averages appear. 

The third step in the process was to eliminate 
all items from further consideration to which a 
er than 85°% responded in the 
same way. In Table 5 the numbers of items in 
cach classification eliminated because of ex- 
treme splits are noted. Among the 32 items 
which were climinated because of bad splits were 
6 which had been included in the Pensacola 7 
Scale, Inasmuch as the items of the Z Scale had 
themselves been screened. for bad splits, this 
result may seem anomalous, All, however, of the 
6 items had had borderline splits in the analysis 
for the Z Scale. By chance fluctuations alone a 
few items with near-elimination splits in the 
earlier analysis would be bevond the 85^", cut- 
off point in the present analysis. Оп the other 
hand, however, none of the 6 items had splits 
beyond 90% in the majority response. Excep- 
tions for these items could have been made. To 
be perfectly safe. however, the cutoff was applied 
without exception. 

After the elimination of items with extreme 
splits the total of 75 items was cut to a work- 
ing pool in cach classification which varied 
between бұ and jo items. These reduced pools 
are the “working pools” of Table 5, where the 
average interitem correlation for each is tabu- 
lated. In all four classifications the average cor- 


majority gr 


10 MARSHALL В. JONES 


relation is somewhat reduced. The items with 
extreme splits tended to be strongly correlated 
with the other items of their kind. Thus, in 
eliminating these items a certain amount of 
homogeneity was sacrificed. Actually, however, 
items as extreme as those eliminated cannot, de- 
spite their correlations, contribute much to the 
variance of the final scale. The fourth step in 
constructing the Z Survey was to determine the 
average item correlation for each of the remain- 
ing items with the other items of its working 
pool. The effect of this procedure is to inform 
us as to which of the items of a given pool 
have the most in common with the pool as a 
whole and which the least. 

The fifth step in the construction of the Z 
Survey was actually to select the final items of the 
four scales. Three considerations guided this 
selection: (a) all four scales were to have the 
same number of items, (b) this number was to 
be as small as possible consistent with the re- 
quirement that (c) all four scales have before 
crossvalidation projected levels of homogenity? 
equal to or greater than +90; since the levels 
projected are based on results in which chance 
was exploited the levels upon crossvalidation 
might very well sink below +.90. From pilot ex- 
plorations the number 49 appeared to be the 
smallest round number that would meet these 
requirements, Accordingly, the best 49 items in 
each classification, ie, the 49 items with the 
highest average correlations, were determined, 
The average correlation of each of thes 
with the other 39 “major” 
tained for all four pools. Th 
of each of the remaining “minor” items with all 


40 major items was also obtained for each 
Not every one of the 


e items 
items was then ob- 
е average correlation 


40 highest averages when these averages ex. 
tended over all the items of the working pool, 
major and minor alike, Against just the major 
items a few of the Strongest minor items en- 
joyed higher averages than did а few of the 
weakest major items. To complete the construc. 
tion of the Z Survey we had only to adjust for 
this overlap. In no group were there more than 
three minor items added and, hence, no more 
than three major items dropped, In the ap- 


The levels of homogeneity were projected 
through the Spearman-Brown formula, 


in which the average interitem Correlation was 
taken as Ty. 


pendix the items of the Z Survey appear, and the 
classifications are noted. In the third row of 
Table 5 the average within-group correlations 
of the four scales are noted, In the fourth Tow 
the levels of homogeneity anticipated by appli- 
cation of the Spearman-Brown formula are also 
noted. These values have a minimum of +90 
for Hostility, . 
The sixth and final step was to determine 
how well the levels of homogeneity anticipated 
from the construction of the Z Survey held 
up in an independent sample of cadets. Accord- 
ingly, the Survey was administered as it appears 
in the appendix to a fresh sample of 230 cadets. 
The Hoyt reliability coefficients were then 
calculated for all four scales, The results appear 
in the last row of Table 5. As was to have been 
expected, all four coefficients ате lower than 
the estimated values in the original sample. On 
the other hand, they are all substantially homo- 
geneous, 


Before turning to the systematic prop- 
erties of the Survey scales, a word is in 
order about their names. The titles, ре 
pendency, Rigidity, etc, are not in- 
tended to describe or predict their rela- 
tions to external measures. The names 
are intended only to describe the content 
of the scale. 

To examine the question of retest rc 
liability the Z Survey was administered 
to а new sample of 218 cadets, All sub- 
jects were readministered the Survey 
after 24 hours, Interestingly enough, the 
four peripheral scales order themselves 
in the same Way on retest reliability 25 
on interna] consistency. ‘The least reli- 
able scale Was Hostility, +.83, and the 
most reliable was Anxiety, +.89- Pe 
pendency and Rigidity fell in betwee? 
these two extremes with coefficients © 
+85 апа +.87 respectively. Thes 
were the only retest data collected on m 
four periphera] Scales; по 4-week dat 
were collected, 

The Pensacola Z Scale was not fakable 
either in the sense of mean change : 
in the sense that More intelligent cade d 
can "fake better" than less intellige? 
cadets, Nevertheless, all four of its СО 


- 
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ponent clusters are fakable, and in both 
senses. Тһе manner, however, in which 
the peripheral scales are faked makes 
quite clear the reasons for the nonfak- 
ability of the Z Scale itsell. 


In examining the fakability of the Z Scale, 
two different designs were used. In the first 
design, fakability in the sense of mean change 
Was examined with two distinct samples of 
cadets. In the second design, a single sample 
Was used to examine fakability in the sense that 
the ability to fake was intelligence-related. To 
examine the fakability of the four peripheral 
Seales. only one design, involving two distinct 
Samples, was used, The first s mple was the same 
“imple of 230 cadets which was used in examin- 
d the heterogeneity and intercorrclations of the 
Survey scales, This mple, was, of course, ad- 
S aed the Sur under normal instruc- 
cadets ie second sample was a group of 221: 
4 ger RS Ei меге administered the Survey under 
same d oy The actual instructions were the 
fakat аз had been used carlier in testing the 

ability of the Z Scale. 
eral ави and variances of the four periph- 
Tat x es under normal and fake sets appear in 
Git AS far as the mean differences are con- 
bur ланып evaluation is unnecessary. All 
"confor, erences are highly significant. The two 
are Ellen scales, Dependency and Rigidity, 
toward i toward heteronomy, ic. they are faked 
бе OE and toward rigidity, The two 
toward ны scales, however, are faked 
anxiety a ; onomy. Anxiety is faked tov 
hostili, poa hostil ty ds faked d non- 
showed n lis split explains why the Z Scale 
a set to Г substantial change in its mean under 
опе w аке, One-half of the scale was pulling 

"ау while the second half was pulling the 


TABLE 6 

TANDARD DEVIATIONS OF THE 
AL SCALES OF THE Z SURVEY 
NDER NORMAL AND FAKE SETS 


————Є—— T Е 3 p 
Scale Normal Set 
(N= 230) 
D 24.92 29:22 
6.48 meat 
R 
$3.1 20:22 
6107 ме 
А x4.do 9.43 
9.87 5.83 
H 17.69 2:92 
Gas 6.24 


TABLE 7 
CORRELATIONS BETWEEN THE ACE AND THE 
Four PERIPHERAL SCALES OF THE Z SURVEY 
UNDER NORMAL AND FAKE SETS 


Scale 


Normal Set Fake Set 
N= 230) (1 
= „ай E —.o1 
М = LR = 003) 
2 .07 = Es 
H +09 —.07 


a The normal and fake coefficients for De- 
pendency and Anxiety are significantly different 
at the .os level. 


other, The fact that the Z Scale changed, though 
without significance, in the direction of heteron- 
omy is also intelligible. There are more de- 
pendency and rigidity items in the Z Scale than 
there are anxiety and hostility items, 

The subjects of both samples were adminis- 
tered the ACE. In consequence, the correlations 
of the four peripheral scales under a normal set 
and under a set to fake were available. In Table 
7 these correlations are set forth. In Dependency 
and Anxiety the difference between the normal 
and the fake coefficients is significant beyond the 
.o5 level. At first sight there seems to be no 
consistency to the coefficients of Table 7. There 
is, however. The more intelligent cadets under 
a set to fake consistently move further toward 
the fake direction than do the less intelligent 
cadets. In. Dependency and Rigidity the more 
intelligent cadets under a normal set are less 
dependent and less rigid. Under a set to fake 
there is virtually no relationship between either 
scale and the ACE. In consequence, the more 
intelligent cadets have taken longer strides 
toward dependency and rigidity than have their 
less intelligent classmates. They have moved 
further under a set to fake toward conformity. 
The more intelligent cadets were slightly more 
anxious and slightly more hostile than their 
fellows under a normal set. The “conformist” 
direction is toward nonanxiety and nonhostility, 
Under a set to fake, the more intelligent cadets 
move further toward nonanxiety and nonhos- 
tility than do the less intelligent cadets. These 
results explain why the Z Scale showed no 
change in its relationship to the ACE under a 
set to fake. The more intelligent cadets move 
consistently in the “conformist” direction. The 
7. Scale, however, is scored so that the changes 
in Dependency and Rigidity are nullified by the 
opposing changes in Anxiety and Hostility. By 
the former pair the more intelligent cadets 
should become relatively more heteronomous; by 
the second pair they should become more auton- 
omous, The two tendencies balance out, and the 
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TABLE 8 


S t ENS OF THE Four 
STRIBUTION PARAMET OE тие. Foy 
goce SCALES or THE Z SURVEY 


wel H 
Measure | D R A _ M 
Е t | 2 92 23.17 14.90 17 бо 

с: | 6.40 6.07 7.63 6 25 
I —.30 == В -41 50 
РА | =  —:03  —.s58 23 


more intelligent cadets end up approximately as 
they were originally, 

The correlations of the four periph- 
eral scales with the ACE under a normal 


set are not without significance in them- 


selves. Dependency and Rigidity relate 
negatively to the ACE while Anxiety 
and Hostility relate slightly positively, 


There appears to be a tendency for less 
intelligent cadets to be more conformist 
and for more intelligent cade 
so. 


ts to be less 


In the homogencity sample of 290 
cadets the distribution 


Parameters of 
the four peripher 


al scales were calcu- 
lated. The results are set forth in Table 
8. As would be expected, the means of 
the two conformist seales аге greater 
than 20, while the mean 


s of the two non- 
conformist scales are less than eo, The 


variances range themselves jn the same 
order as the homogeneity levels of the 
four scales, Again, this Was to have been 
expected. Test variance and 
neity have а common 
average interitem correl 
ner in which the four 
is also typical. The 


homoge- 
item parameter: 
ation. The man- 
scales are skewed 
two conformist 
scales are negatively skewed while 


the 
two nonconformist scales 


are positively 
skewed. With the exception of Hostility 
all scales are platykurtie. Hostility tends 
without significance toward le 


Piokurto- 
sis. There does not seem to be 


any very 
obvious explanation of this tendency, 
In its development and study 


the 66 
items of the Pensacol 


а Z Scale were ad- 


ministered alone. After its incorporation 
into the Z Survey as Heteronomy the Z 
Seale became the first 66 items of a 197- 
item questionnaire. The change in ~ 
text which this incorporation. involve 
could conceivably alter some of the peep 
erties of the Z Scale. The matter docs 
not seem very serious since the order and 
priority of the Z-Scale items have ND 
been changed. Indeed, all that has 
changed is a set: the cadet in i king the 
7. Scale knows he is taking a G6-item in- 
ventory; in taking Heteronomy he is ui 
ing a 197-item inventory. To Noii 
however, some of the basic properties © 


7% : see if they 
the Z Scale were examined to see if they 
generalized to Heteronomy. 


П 
Applying the Hoyt analysis to the йм. Ss 
items of the Survey we obtain an internal only 
sistency coefficient of 4-609. This result uer] 
slightly less than the value for the Z Seale its we- 
We may, therefore, condude that the попов 
neity or, if you like, the heterogeneity of m 
Scale, has not been disturbed by its incorporati 
into the Z Survey, 
In 


1% 
s18 cadet 
the 24-hour retest, sample of 21 
the 


is “This 
‘liability of Heteronomy was » 


; original 
; т 4 ` origin 
value is only a trifle lower than in the orig 
scale. 


The distribution of Heteronomy is also bie 
sentially like that of the original Z Seale. m 
sample of 290 cadets the mean was : 
the standard deviation 5.78. In the same 
there was a definite negative skew, £1 
and a slight platykurtosis, g: 
sults are all typical of the original scale- seron- 

In the “fake” sample of cadets Hete 


so virtually 
omy had a mean of 35:56. This value is virt! ihe 
identical to the value which obtained x not. 
homogeneity sample, Heteronomy docs а 
therefore 


 аррсаг to change its mei 
Set to fake any more than does the 7 
The correlation between the ACE Ма 
momy in the sample of 230 cadets ee 
The Correlation between the same two mc Aven 
in the f, e 
though —09 is an unusually low value ier tle 
relationship between heteronomy and ee 
aptitude there is no significance to the diffe a 
between it and the fake coefficient. of Sade 
like the 7, Scale, docs not ар} 


c 
hang 
i ; an chi 

cither in the sense of mea cadet 


n under 


эе 21. 
ake sample of 221 cadets was —- 


Heteronomy, 
to be fakable 
or in the 


intelli t 
Sense that the more intelligen 
can fake h 


etter than the less intelligent- 


—— w 
—— — 
———' 
ت‎ 


ГИК PENSACOLA Z SURVEY 19 


In Table g the intercorrelations of the 
five scales of the Z Survey are set forth. 
As is clear from the table, the two con- 
formist scales account for a much larger 
part of the variance of Heteronomy than 


TABLE о 
INTERCORRELATIONS or THE FIVE SCALES 
or THE Z SURVEY 


a D R A 11 
| Е d 
M у 


denotes Heterononty. 


do the two nonconformist scales. Also 
Worth noting are the negative correla- 
oe the conformist and the 
в Mormist scales, At first, these rela- 
“ons may seem odd. However, in their 
oe we must remember that a 
5 score on the Anxiety Scale, for ex- 
е; does not mean that the subject is 
anxious; it means he describes himself 
E esse ‘Though they may be related, 
mum us and to desc ribe oneself as 
this inn Pe two different things. With 
ый тн the negative relations may 
ample p p cita A person, lor eR 
Would سن‎ describes himsell as rigid 
self ae пор scem likely to describe him- 
S00 AS anxious too. More broadly, cadets, 
Ms апу rate, may ascribe to themselves 
eo or nonconformist traits but 
У tend not to ascribe both. 


SUMMARY 


In the attempt to free the measure- 
ment tendency from 
political particularity, it was resolved to 


of authoritarian 


use materials as purely personal as pos- 
sible. From such materials the 66-item 
Pensacola Z Scale was then constructed. 
Phe Z Scale correlates +.43 with the 
California F Scale, has a 24-hour retest 
+.8-, is unfakable and 
heterogeneous (+ з). The 
gencity ol the Z Scale stems from the 
presence within it of four distinct clusters 


coefficient. of 
hetero- 


ol items, which may be described as "de- 
pendency.” “rigidity.” "anxiety," and 
"hostility" items. In the Pensacola Z 
Survey these four clusters are expanded 
into four corresponding scales. These 
four scales, together with the central or 
basic Pensacola Z Scale, constitute the 
five scales of the Z Survey. The homo- 
geneity levels of the four peripheral 
scales range between .82 and .go. The 
»y-hour retest coefficients range between 
83 and .89. АП four scales are [akable. 
The directions in which the four scales 
are faked divide them into a conformist 
pair, Dependency and Rigidity, and a 
Anxiety and Hos- 


nonconformist pair, 
tility. The two conformist scales corre- 
late negatively with the two nonconfor- 
ally, the systematic prop- 
le are not substantially 


mist scales. Ё 
erties of the Z $ 
altered by its incorporation into the Z 


Survey. 
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APPENDIX 


In 29 of the first 66 items and in all of the 
remaining 131 items, one of the two statements 
in each item is followed by an upper-case letter 
in parentheses. The statement so indicated is 
the heteronomous member of the item, and the 
item belongs to the Dependency, Rigidity, Anx- 
ietv, or Hostility Scale according as the letter is 
D. R. A, or H, There are 40 items in each 
scale, The first 66 items constitute the Heter- 


onomy Scale (Pensacola Z Scale). Those items 
among the first 66 which are not part of a pe- 
ripheral scale are marked with a lower-case letter 
(d. г, a, or h) which indicates the classification 
of the item: dependency, rigidity, anxiety, or 
hostility. 'The 29 items in the Heteronomy Scale 
which do belong to a peripheral scale have, of 
course, the corresponding classification. within 
the. Heteronomy Scale. 


THE PENSACOLA Z SURVEY 


In this test you will find pairs of statements 
having to do with personal characteristics. One 
member of the pair is labeled A and the other 
В. You are to select from cach pair the statement 
that BEST describes you. Then indicate the 
Statement you have chosen by making a hea 
black mark between the lines under the letters 
А or B (but not С, 1), ог Е) on your answer 
Sheet. Consider the example shown below. 

1. A) You are attractive. 
B) You are strong. 

If you think You are strong describes you 
better than You are altractive, you would put a 
Mark under B on your answer sheet. Your 
answer sheet would then look like tl 


АВ С 
Ai FETE d» a 


If you marked B on your answer sheet, it 
Would not nece rily mean that you are ex- 
tremely strong or that you are not attractive. 
It would mean that ON THE WHOLE, You are 


arang describes you better than You are attrac- 
ive, 


Be sure that you select one statement from 
ету pair. You are not permitted to omit any 
Pair of statements, Mark your answers on your 
answer sheet starting with number 1 and con- 
tinuing through number 66. You should finish 
the test in approximately 15 minutes. 

You are too friendly for your own good. (r) 
B) Your opinions are often incorrect. 

Taking advantage of a person sexually makes 
you feel bad. 


В) You have no scruples in sex. (Н) 
3 A) You are anxious. (а) 

B) You are conceited. 
“ A) To you life is a jungle. (A) 


B) To you life is a bowl of cherries. 

You day-dream politically. 

B) You don't formulate opinions about issues over 
which you have no control. (d) 


6, дб Mes 
A) In political activities you confine your efforts 
B to group action. (d) А И 
) In political activities you frequently indulge in 


ЖА individual endeavor. 
) You like a tightly organized group. 
You like a loosely organized group. 


(d) 


Кез in your life. 
you have made out 


8. A) You haven't made any m 


9. A) There are some people you could never feel 
for. (H) 
B) Sometimes you feel a real compassion for ev- 
eryone. 
10. A) You like instructions to be specific. (d) 
B) You like instructions to be general. 


11. A) You are sexually appealing. 
B) You are faithful. (D) 
12, A) You are responsible for most of your troubles. 


В) You sometimes get confused without any rea- 
son, (a) 
13. А) You frequently laugh at yourself. 
B) You don't like your favorite habits ridiculed. 
(г) 
14. A) You frequently get away with murder. 
B) People often blame you for things you didn't 
do. (a) 
15. A) You are not attracted to prudish people. 
B) You are not attracted to unkempt people. (г) 
16. A) You want badly to "belong." (D) 
B) You don't care whether you “belong” or not. 
17. A) You like a clean, neat house. (R) 
B) You like good food, 
18. А) You can never forget that love is more than 
just sex. (R) 
B) You can take pleasure in sex as sex. 
19. A) You are always on the lookout for new ways 
of attacking a problem. 
B) In general, you find the tried-and-true meth- 
ods work best. (r) 
20. A) You are rebellious. 
B) You like discipline. (D) 
21. A) You don't like to gamble on getting a good 
break. (a) 
B) You usually figure on getting a good break. 


22. A) You get more credit than you deserve. 
В) You get less credit than you deserve. (а) 
23. A) You get into scraps you didn't start. (а) 


B) When you get into trouble it is almost always 
your fault. 
24. A) Most everybody lets you know directly what 
they think of you. 
B) Some people are secretly trying to get the bet- 
ter of you. (h) 
25. A) You positively like to be different from your 
immediate associates. 
B) Being different from your immediate associates 
makes you uncomfortable. (D) 
26. A) N are either your friends or your enemies. 
т. 
В) People аге rarely either real friends or real 
enemies, 
27. A) Your hardest battles are with other people 
rather than with yourself, (h) 
B) You are cocky. 
28. A) You could like anyone if you tried, 


Oc " Research! 
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B) There are some people you know you could 
never like. п 
А You are forgetful. А 
Be 1d iS memory, (R) " 
ES There are some people you would like to te 
зо. 4 
of. (h) | 
B) You are occasionally taken in, 
A) People criticize you unjustly. (a) 
° "B) People give you more breaks than. you deserve, 
2, А) You are charming. 
= р) You are firm and resolute. (R) 
A) Disappointments affect you so little that you 
ubl 1 195349 | 
B) Your daydreams are often about things that 
can never come true. (A) h à 
34. A) You would like to counsel a friend on his per- 
i sonal problem, —— | 
В) You would like to give first aid to a f 
(d) К 
35. A) You collect things. (R) 


B) You lose things. _ 
36. A) You like haphazard living. 
"^ В) You like routine. (К) 


37. A) Stuffed-shirts amuse you. А 
В) Stuffed-shirts get under your skin. (h) 
38. A) You keep calm in an emergency, 
B) You can obey orders, (d) 
39. A) You are difficult to please, 
B) You like to do favors. (D) 
40. A) You are aware of dripping water in the 
kitchen, (г) 
В) You аге not observant, 
41. A) You don't mind a coward, 
В) You can't stand a coward, (h) 
A) You just can't stay m 
you should, 
B) There are some people 
apart. (Н) 
43. A) You admire Spontaneity in people, 
B) You admire efficiency in people, 
44 A) You don't particularly like to march, 
B) You like to march with a group 
to belong to. (d) 
45. A) You need someone in whom you can confide 
completely, (D) 
В) You are selfish, 
46. А) You play fair. (D) 
D) You are an individualist, 
47. A) There are some magazines to which 
ticularly turn. for the subst 
political ideas, (d) 
D) Your political ideas tend to he peculiar to your- 
self, 
48. A) You can't help feeling аша, 
who hold important opinii 
ferent from yours, (h) 
B) You like a lot of people who disa; 
violently on important issues, 
49. A) Your interest in general principles occ; 
gets you up in the clouds, 
B) You are а stickler for precision, 


50. A) You have felt so sorry for someon 
cried, 


B) You have gotten so m. 

51. A) Yours is a quick ane 
B) Youarestern. (Ну 

2o A) You are independent, 
`7 B) You are loyal. (D) 

53. А) You are talkative. 

и В) Often уоште sure you've fo 

important. (А) 

54. A) You would be happier if you felt more se 
(a) > 

B) You would be happier if 


ad even when you think 


you would like to take 


(R) 


you feel proud 


you par- 
antiation of your 


gonistic to People 
ons radically dif- 


gree with you 


asionally 
(R) 
е you have 
ad you cried, (h) 


1 ready Sympathy, 


Preolten Something 


cure, 


YOU Were less muj 
5s. A) You never change your Казу, beliefs, “rile. 
(0 B) AML your beli fs are open to debate, 009 


56. 


64. 


66, 


68, 


69. 


A) You follow your conscience, В 
P YA ae EEA Staats BTR gua HALLO 
A) e very proud of your membership in 
some zroups. (d) 
B) You don't go for groups. 
A) You are indifferent to most people. 
B) You like or you dislike people. (r) T 
A) You don't worry about physic, bans A Ж 
B) Sometimes you figure you're a sure thing 
ulcers. (а) 
A) You are dogmatic. 
1) You are sloppy. : : 
E nee are =й people you admire so i 
you would not question their opinion. 
B) You don't admire anybody very much. am 
Concerning your past ad you figure, 
1 did it, it can't be too bad. a 
If ы your life to live over, there ШТ 
с a lot of things you'd do differently. ү 
Imire careful, rigorous thinking. ы (r 
Imire brilliant, penetrating ишш, 
A) The details of life are important to you. 
B) You are often thoughtless. 
1 € well coordinated. (г) 
new opinions. 
e self-confident. 
B) You are а good Joe. (A) 


(R) 


A ^ cason. 
A) You are often depressed for no good r 

(A) “th live 
B) You always feel life is very much wort 

ing. If 
A) You would like to work by yourse ТОА 
B) You would like to work in a good orga 

(D) member 
A) You have often felt sick when you nane 


ипеопе. 
about people you hurt, 
a 


how badly you have treated 
B) You don't worry 


they deserve 


when 


A) You could use some good counsel. (D) 

B) You could do with a pood friend. 

A) You are eccentric f with a great 

B) You could easily identify yourself with а ET 
cause. (D) 

A) You like variety, R) 

B) You always finish a job you start. ( 

A) You can be squeamish, 

B) You can be brutal, (OD 

A) You are thick skinned, 

B) You are easily slighted. (A) +f you were 

A) Yon would be more successful if yor 


less optimistic. " had more 
You would Le more successful if you Ma 
confidence, (A) 
You are a fault-finder, 
You often have the blues, er of à 
"ou could see yourself as a loyal follow 
Rreat тап. (D) 
You аге а mediocre team-man. А 
i re touchy about some things. 
ire not е insulted. 4 
Being yourself means being different 1 
Being yourself has nothing to do with 
different. (D) 
You are orderly, 
You are cheerful, 
А You are sensible, 
3) You are 


(Н) 


(А) 


eing 


(R) 


(R) 
y ‘A Spontaneou 

ou feel ng necessit 
B) Distiking someone ma 


AS someone! 
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'eople 


aD ple 


to like people. forta 


kes you û uncon! 
5 disliking yo. 
Can get very nearly anything 
n) y if they ery, 

M сан" stand « 
\ hen you hurt 


feel no uilt 
B) You i guilt, 


jo 
from Y? 
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inadverte 


е youl 
dramatic , 


Someone 
(1) 


almost as badly when you M 
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art! 
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102 
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one malverrently as when yon meant to hurt 
them. 

You are methodical. CR) 

You ate curiis. 

Nobody 


over on уоп. (H) 
tating experience quickly. 
ful person. 


purs апу 
You get ove 
You are a det 


You are an etteient person. (R) 
otten reject your suggestions not on 
ments but because it was уоп who 
made them. (А) 
People as a rule give you credit for more 
Windom than. you possess, 
Yon would describe your thinking as 75 
жый" 
You woubl describe your thinking as “accu 


tate," CR) 

Your ambition is mostly personal. 

Your ambition is not so much for yourself as 
ior certain ideals whieh you hold. (D) 

At times you think you are no good at all. 
CN) 

You never seriously doubt your own abilities. 

You tend to do a lot of “soul-searchin 63) 

In general you concentrate pretty much on the 
problems in front of vou. 

Vou can't stand people who are politically and 
милау denoerant. (HD 

You don't place much emphasis upon how ig 

огл a man is m polities and social affairs. 

There are people who are an inspiration to 
you. (D) 

Mostly vou are inspired by ideas, 

You never say anything nasty just to be mean, 

Lots of times you just can't help ing what 
8 dirty soando you think someone is. CHD 

You notice when window shades are not drawn. 
(R) 

You are often neglectful. 

Recit to yourself your own 
way of calling out the best in you. 


name has a 


Those thines whieh symbolize the groups to 
which yon belong call out the best in you. 
(D) 


You iry to be creative in your approach to 
things 

You try to be sound in your approach to things. 
(R) 

You don't mind apologizing as long as you're 
even partly wrong. 

You find it hard to apologize, particularly when 
the other fellow is as much to blame as you 
are. (H) 

en when you are with your friends you 
feel lonely much of the time. (A) 

Loneliness for you is just a feeling, which 
doesn't 1 

Nervousness keeps you from being as popular 
as you should be. (A) 

You are very popular as you ar 


long 


You try to keep a tight rein on yourself at all 
times, (R) 

Much of the time you rush things v 
thinking. 

You like parties where people just get together 


and let things develop 

You like well-arranged parties. (R) 

Tn many things you are fickle, jumping from 
one thing to another. 

You always stick to a plan onc 
cided upon it. (R) 

Lots of times people “on the outside" see your 
problems more clearly than you do, 

Tf there is anything which gets you, it 
who thinks he knows your problems better 
than you do. (H) 


you have de- 
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it way would be. 

You feel sympathetic toward people who tend 
to hang on to their griefs and troubles. 

You have no sympathy for people who agers 
vate their own problems. (H) 

You can't forget certain failures and humilia 
tions in your рам. (А) 

As you look back on your life you can see 
things have been for the good. 

You can feel, not just intellectually accept, the 
power of the people. (D) 

"The only power which you really feel is the 
power within yourself. 

You prefer work which requires close atten- 
tien to details, (R) 

You'd go nuts doing precision work, 

You are responsible to other. people, those you 
love. those who depend on you, and so on. 
(D) 

You are 
ideal 

Often times you 
door open, a ciga 
mailed, and so on. 

You check and recheck things to make sure 
vou have taken care of them. (R) 

You have often had trouble sleeping. 

You have always been a sound sleeper. 

You like the aloneness and independence of 
a bustling cit 


responsible to yourself, 
ambitions, and so on, 
i that 
ette lighted 


your own 


you've left a 
a letter un 


(A) 


You like the atmosphere of a crowd of people 
who meet for a common purpose, (D) 
There are days wh you can't make even 


(А) 

You sometimes mess up a small thing but you 
rarely go wrong on the big ones. 

You like a person to be modest even if he is 
gool. (D) 

You like a person to say he is good, provided 
he is. 

You feel tha person should dedicate himself 
to something, (D) 

You are inclined to be self-centered. 

Your first impressions of people are usually 
correct, (H) 

You find that as you get to know a person, you 
get to like him. 

You have solved most of your personal prob- 
lems. 

You seem to have a lot of unsolved personal 
problems. (A) 

To see any other person suffer would upset you 
very much, 

To inflict pain on some people would not be 
altogether unpleasant to you. (H) 

You wake up fresh and rested almost every 
morning. 

There are many days you dread going through 
with. (А) 

You used to keep a diar: (R) 

Oftentimes you forget important dates in your 
life, like the day you graduated from high 
school. 

Your emotions are 


not easily aroused. 

You are ily excited or depressed. (A) 

You feel “at sea” much of the time. (A) 

Psychologicfily you are always in good shape. 

You often make fun of other people. (Н) 

You often make fun of yourself. 

You are suspicious. — (II) 

You are trusting to а fault. 

You are е y-go 
things. 
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B) You try to conserve and plan for the future. 
к 

A) You е for the general good first. (D) 

B) You live for yourself first. 

A) You have had periods when you lost sleep 
worrying. (A) 5 

B) You simply aren’t a worrier. А 

A) You often ask people for advice. _(D) 

B) People often imply that you think you are 
better than you are. 

A) You forgive people easily. А 

В) When people apologize, you can't help feeling: 
actions speak louder than words, (H) 

A) You like to feel a member of a good, solid 
group. (D) 

B) For some reason, you are 
groups, 

A) You think of yourself as first of all an individ- 
ual person, 

B) You think of yourself as first of all a social 
being responsible to society and those who 
think like you. (D) 

A) Sometimes you change your way of talking 
and the things you say in order to win 
popularity in a group. (D) 

H) You've yet to find à group of people who were 
worth much effort to get in with, 

А) If someone hurts you, you try to get even with 
him. (H) 

B) You are even-tempered, 

A) Your emotions rarely get in 

B) You have a lot of “pent-up” 


a bit allergic to 


in life as well as games, 
chance of success is “to 


ose to your chest,” R 
A) You are self-reliant, B 


B) You can be depended оп. 

А) One of the nicer thin 
you don't have to 
time, 


B) Ws dante like to visit in a Poorly kept house, 


85 about vacations is that 
be well-groomed all the 


A) There are a lot of days when no matter how 


hard you work you just don’t seem to ac- 
complish anything. (A) 


В) You always manage to get somethin, 
& done, 
A) You have confidence in yourself, 


B) You have confidence in things you and those 
like you stand for. (D) 
A) You tend to be forgetful ín m 


at you haven't 


dequate to carry 
et for yourself, 
y of you when you are 


ing out 


B) People talk criticall 
around. (A) 
A) You admire the efficienc: 


y of you when Уой are not 


Y of a well-run organi- 


zation, (R) 
B) You admire the drive of а well-run organiza. 
tion. 


A) You don't care much for "lone wolves,” 

B) You are a “lone-wolf” yourself, 

A) Some of your frends think that your i 
impractical, if not a bit wild, 

B) You stick to matter of fact as тис Э 
sible. (В) E Жы 

A) There аге а lot of people you know who don’t 
like you very much, 

B) You lack self-confidence, (A) 


(D) 
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You are often nasty. 

You are very conside 

You always work tow 
defined results. (R) | : 

Much of the time you aren't precisely sure 
what you're working towards. " 

Physically, you are “in the pink" almost а 
of the time, 

You have trouble concentrating. (А) {сау 
You enjoy confused and noisy but basically 
friendly debate. ere 
You like a smooth-running organization. 

You like variety. 

You like order. (R) 

You are forthright. (R) 

You are carefree. 

ye “feel ТАШЫ: sorry for the habitual com- 
plainer. | 

Whining, complaining people infuriate you. 
H в 

М resolve to do a thing you always 
carry through, (R) 8 е 

Concerning your own resolutions you figure: I 
made it, I can break it, ite, 

Disliking people makes you feel uncomforta 

There are some people that you dislike En 
much that you are Шве), when they E 
what is coming to them. (H) чар 

You are баа your personal habits and 
dress. (R x 

You tend Is NM your personal appearance. 

You like individual sports. 

You like team sports. (D) £ 

You tended to be solit as a child. it 

When you were a child, you беоне. pr 
mang that stuck together through thick 4 
thin. (D) . "T" 

You admire decisive, forthright action for its 
own sake. (D) 

Sometimes you like to be alone. 

You don't belittle people. $ you. 
You hate people who are condescending to У 
(1) 


ж. 
rd tangible and clearly 


(D) 


a 


You are a slob, 
You are a snob. (Н) 
Sometimes you feel it is necessary to 
someone for his own good. (Н) И пеопе 
You would have a hard time hurting som vod. 
even if you thought it might do some £ 
You are a perfectionist. (R) 
You like “big” ideas. " 
You don't think much about your pas! ‘can't 
There are some things in your past you 
Seem to get out of your mind. (A) oti 
You often find that people who are antone 
to you would really like to be friend t ace 
You often notice, beneath a person's SU 
friendliness, а deeper hostility. (H) h” 
In most things you have “a golden Кустуйїп& 
You have days, even weeks, when ever) 
you do seems to go wrong. (А) пе. 
You hate to force your opinion оп ay ресе 
Lots of times you just can't argue with P (Н) 
you have to tell them what's what. 
You are harsh in your judgment of 
(н) le. 
You are lenient in your judgment of penn: 
You take jibes and insults in your stri орі. 
Оз аге more "sensitive" than most Ре : 
(A) 9 
You sometimes wonder what all the struggle 
life is about anyway. (A) 
ou don't worry much about “the stru Н 
life” but stick to the problems at hanc. 
You are almost always alert. (R) 
Sometimes you just don't pay any atten 
What's going оп, 


hurt 


istic 


people- 


ggle of 


tion 10 


180, 


186, 
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You enjoy the excitement of a crowd. (D) 

You rather dislike crowds. 

Generally you let a dirty crack go without re- 
taliatin 

You don't believe in “turning the other check.” 
ap 


In group 


ties vou ket more than your 


m. 

oup activities you are sometimes glossed 
over, (A) 

You сап be “hard (H) 

You are a sucker for a good “hard luck story." 

Sometimes you do things just “to show" some 
body. (ID 

You make a pomt of not throwing a fellow's 
mistakes up to him, 

Yon have male more progress in life than the 
great majority of people vour age. 

Sometimes you feel as if life is passing you 
hy. CA) 

You enjoy social gatherings just to be with 
people. (D) 

You don't enjoy big parties. 

You want to be judged solely өп your own 
merits, 

You don't mind being judged by your afilia 
tions. (D) 

People ignore you. (A) 

People “put you on a pedestal.” 
You find it difficult to break with familiar and 
pleasant surroundings. (A) 
You feel at home almost anywhere. 
You have often been criticized 
ciently humble. 

You place great emphasis upon being a true 
friend. (D) 

You live from d ay to day. 


as not sufi- 
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You try to plan your future in great detail. 
(R) 

When you were a child you didn't care to be 
a member of а crowd or gang. 
You have alw been a “joiner.” 
People often resent your giv 

structive criticism, (Н) 
You don't give advice unless the other guy 
asks you to, and not always then. 
You enjoy parades but you have no desire to 
be in them. 
When you see a parade go by, you feel like 
going out and marching with them. (D) 
You are pretty self-satisfied. 
You long to be at peace with yourself. 
You are exceptionally healthy. 
You are moody. (А) 
You often try to “fee 
one else's point of view. 
You treat people according to what they de- 


(D) 
g them 


con- 


(A) 


into some- 


serv aD 

There are people in the world for whom you 
feel nothing but hatred. (H) 

You don't hate anybody 


You tend to be absent-minded, 

You are thorough. (R) 

Sometimes you say hard things about people. 
(H) 

You bend over backwards not to talk anyone 
down, 

You often find yourself questioning the loyalty 
of your friends. (A) 

The loyalty of your friends is not a matter of 


great moment to you. 

People sometimes criticize you for being 
“soft.” 

It does not bother you particularly to see ani- 


mals suffer. (H) 


а س‎ 


Е 


Vol. 71. Хо. 24 


Whole No. 453. 1957 


Psychological Monographs: General and Applied 


Effects on Clients of a Reflective and a Leading 
Type of Psychotherapy’ 


Јоко D. Аину, Doxatp H. Ford, BERNARD G. GUERNEY, JR., 


\хр Lovrnt 


GUERNEY 


With an Introduction by 


WILLIAM 


SNYDER 


Pennsylvania State University 


IN PRODUCTION 


T Ur FOUR integrated studies described 
: in the pages following this introduc- 
Чоп are interesting in several respects. 
First, the design was unique. The re- 
Searchers have compared a relatively 
Nondirective method of treatment with 
5 somewhat 
Order to 


interpretive method, in 


determine whether one ap- 
Proach induces more resistance in the 
Client than the other, whether one pro- 
C uces more dependency, and whether 
анч iih certain personality charac: 
otia: sd relate better to the therapist m 
ees pi Kup: ка, тап the suum 
itg in я to make this comparison, train- 
Was ! the use of both treatment methods 

усп to то therapists. Then, during 
" Ume interval of one semester, almost 
I of the clients asking for help at the 
Yehological Clinic of the Pennsylvania 
igned 


State ; А 

i e University were randomly as 
о in А B 
these 10 therapists and to the two 


Ure; s : 
ү atment conditions. Each therapist 
Orke . = x в 
dis ked with four clients, using а non- 
lrectis E у 
Clive approach with two and a more 
4 i This 
Ondens, 
‘lions 


report constitutes an integration and 
ation of four interrelated doctoral disser- 
a, 9. 16. 17) from the Pennsylvania State 
Under Sly. The studies were conducted jointly 
ton E supervision. of william ч. Snyder, 
Orlow, and Alec J. Slivinskc. William Ray 

as statistical consultant for the studies. 
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interpretive approach with two. Analyses 
of variance were completed on the data 
from the two samples. This design made 
possible a comparison of the two thera- 
peutic approaches, a comparison of the 
therapists with each other, and an evalu- 
ation of the effects of a particular 
therapist. using а particular therapeutic 
approach. Contributing to the strength 
of the design was the fact that the ex- 
perimenters themselves did not act as 
therapists. 

A second aspect of these studies that 
is of interest is that the investigators 
found it quite possible to classify client 
and therapist verbal behavior directly 
from the recordings, rather than from 
typed transcriptions of the interviews. In 
fact, they felt that their classifications 
were probably more accurate under these 
circumstances, since voice inflections 
were often crucial in determining how 
to code a particular response. 

А third some 
rather surprising reactions by the thera- 
pists. For example, in spite of the fact 
that all 10 therapists seemed quite will- 
ing to participate in the study, and the 
experimenters used considerable tact in 
working with them, some of the thera- 
pists did not observe the limits of the 
study. Four of them decided that “thev 


aspect of interest is 
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knew best" and consequently deliberate- 
ly used a different technique if they felt 
that it was in the best interests. of the 
client. Although several explanations are 
possible, the one that seems most likely 
to the experimenters, in view of the in- 
formation available, is that their per- 
sonal needs were too strong for them to 
be willing to remain within the limits 
of the experiment. Three of the four 
were rather authoritarian in their rela- 
tionship with their clients, and the 
fourth had a Very strong need to be non- 
threatening. Another thing that wasn't 
anticipated is that, although the thera- 
pists could learn quite satisf. 
to play the roles required i 
ment method, this intelle 
tion of what Was expected 
not necessarily 

their personal n 
thought best, 
answ 


actorily how 
n each treat- 
ctual percep- 
of them did 


ttempt to discover 
еп the thera- 
characteristics and 
unsuccessful. Noth. 
of what a therapis 


personality 
clients’ behavior was 
ing is as indicative 


t 
will do in therapy as a Tecording of his 
actual in-therapy behavior! Another 
finding of interest conc 


is that clients seem 


personality factors 
effective therapeut 
tinues to be a challe 
has observed that, 


ате conduciy, 
1С relationshj con- 
nging one, The Writer 


Contrary to his f 
E : ori 

suppositions, it is possible for a thera E 
in-training who has a number of Pist- 
sonal problems to establish 4 per- 


n adequate 


€ to an 
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therapeutic 
clients, 


relationship with some 
The writer would like to mention dd 
brielly some of the results of the fed 
integrated research studies, (a) 1 he darn 
pists consistently rated their clients 5 
having improved more under an inter- 
pretive treatment than under the nou 
directive one, No other change variables 
reflected. differences between treatments, 
The leading treatment produced -— 
guarded verbal behavior. Since i 
therapists expressed a preference for A 
interpretive type of therapy, 79 
searchers point out therapists’ m 
may have been influenced by ped 
therapy preferences, (b) Client 2 
therapy personality character is - 
seemed to be more important in ш heed 
lerpretive therapy than in a ИШИ 
tive one in the way they related B i 
fensiveness during the early Er n а 
of therapy. (c) The therapists уе 
the amount of "guardedness" anc) 
“defensiveness” 
clients, Also, 
a particul 
different 
ness, 


they engendered in е 
а particular therapist ie d 
ar treatment method produces 
effects in his clients in pert 
This suggests that for the beg 


e s . of per 
ning therapist a subtle interaction © rent 
sonality са 


: ty important to the 
ol defensiveness he produces in i 
ents, One might say we have know it 
all along, However, it is of value M i 
experimenta] evidence of this fact 

need to obtain me 
about iis. t 
researchers 
and * 
terre] 


sre specific s 
antalizing result. e nnes> 
observed that ae А 
'Buardedness" in the client " " 
ated and that the one PE ош 
SPonse cannot be considered k^ 

Telerence to the other. In other ,, jf 
When Clients seemed really t° ites 
Volved in therapy, they woule 
Problem, discuss it, and then, 


EFFECTS OF A REFLECTIVE AND 
it is painful to face oneself and modify 
one's sell concept, they would become 
Somewhat defensive or guarded. ‘This 
sort of behavior seemed to occur in cy- 
clical form. As a consequence, the re- 


Searchers were led to the conclusion that 


perhaps resistance, or defensiveness, may 
be a “good” sign, in that it indicates that 
the client is really working on his prob- 
lems. If he never gets into any real prob- 
lems he has little need to be defensive. 
Of course, the "best" therapists would 
be those who would be able to keep this 
client defensiveness at a minimum and 
would not add to it through their own 
Mept behavior. 

Illustrative of how puzzling some re- 
sults can be are the following findings. 
From some points of view it appeared 
SUperficially that the 
method produced more desirable results 
an the interpretive. For example, in 
the 


nondirective 


i nondirective approach, there were 
“ger percentages of client "openness" 
and “covert resistance” while the inter- 


Telive 
үе method had larg 
0 


r percentages 
ii "dependency," "guardedness," апа 
Overt resistance.” Further, analysis of 
the “covert resistance” 
Yealed that in the nondirective method 
12% of these responses were so classified 
Cause the client had made long pauses, 
While in the interpretive method only 
1396 of “covert resistance" responses 
Hi due to long pauses. Also there were 
ess "blocking" and "interrupting" in the 
Nondirective than in the interpretive 
Ve B However, from another point of 
teme hn more interpretive t р 
the ín superat, For example, clients in 
come terpretive therapy tended to be- 
Bites ше positive in their feelings 
Scale d therapy, as measured by a rating 

Completed at the end of the fourth 
ма авай at the end of the eighth in- 

w, whereas clients in the more non- 


responses те- 


therapy 


A LEADING 


TYPE OF PSYCHOTHERAPY y 
directive therapy tended to become more 
negative or defensive. Also, therapists 
were able to hold clients in therapy bet- 
ter in the interpretive situation than in 
the nondirective one. Both 
seemed to have certain aspects which 
produce favorable reactions in some 
clients. 

Another puzzling result was in regard 
to the relationship between therapist 
personality and client "guardedness." If 
"guardedness" scores were considered in- 


therapies 


dependently, the lowest scores in this 
category were with therapists who were 
friendly, uncritical, and took a conver- 
sational approach, rather than focusing 
problems. Also, 
therapists tended to 
more decrease in maladjustment scores, 
while clients of warm, accepting, but dy- 
namically sensitive therapists tended to 
grow worse in adjustment, although this 


on client clients of 


"[riendly" show 


change was not statistically significant. 
It would appear, then, that the therapist 
who is friendly and uncritical and who 
does not focus as consistently on prob- 
lems produces the least client "guarded- 
ness" and the greatest decrease in “mal- 
adjustment," while the therapist who is 
warm, accepting, but also aware of the 
client's problems and his motivation, 
produces more "guardedness" and more 
“maladjustment.” The researchers in 
these studies chose to believe that the 
"guardedness" was a necessary concomi- 
tant of facing unpleasant facts about 
oneself, and that the seeming increase in 
maladjustment was a temporary one 
which would be replaced by a decrease 
in maladjustment as therapy progressed. 

Lest the present studies should tempt 
the reader to make unwarranted gen- 
eralizations, the writer would like to 
mention some limitations of the studies. 
First, although the two therapies were 
very different in some respects they were 
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similar in others. The interpretive ther- 
apy was not extremely so, and the non- 
directive therapy was not completely 
"pure." Second, the treatment. lasted 
only through one semester, with an 
average of about thirteen interviews. 
Third, the therapists were only moder- 
ately experienced. Fourth, the attitudes 
of the therapists were such that they ap- 
peared to have more confidence in the 
interpretive therapy than in the non- 
directive therapy. This lack of confidence 
could have influenced the effectiveness 
of the therapies in subtle ways. 

The results of these four studies make 
us more aware than ever of the need for 
continued research on the nature of the 
relationship between the therapist and 
the client. We need to be able to identify 
the personality characteristics which en- 
able a therapist to function in a maxi- 
mally effective manner, a 
those techniques which 
most rapid and le. 
the client. It ap 
plore client 
havior in rel 


nd to delineate 
will produce the 
ast painful progress for 
pears important to ex- 
and therapist interview be- 


ationship to therapeutic ош- 
comes. In the writer's opinion such re- 


search will be more likely to be useful 
if it is based, at least in part, upon what 


actually happens in the therapeutic in- 
terview. 


NEED AND STATEMENT OF THE PROBLEM 


Over the span of years since the intro. 


duction of Rogers’ Counseling and Psy- 
chotherapy (23), interest and research in 
the area of psychothera 

ly expanded (2, 3, 8, 10, 21, 25, 20, 91). 
However, the need for continued re. 
search and the development of More sys- 
tematic theory is evident from our too 
limited knowledge of the therapeutic 
process. 


Recognizing the need for research on 


ру have constant- 
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psychotherapy, and particularly ше 
need for more comprehensive апа better 
designed research, the writers set gue 
to develop a research project which in- 
corporated a formal experimental de- 
sign, which encompassed numerous vari- 
ables related to the therapeutic process, 
and which involved more adequate 
samples of clients and therapists. The 
fundamental purpose of the project m 
to analyze many different variables anc 
to assess their relationship to uierapy: 
The list of problems investigated in this 
study follows. 


Effects of Leading and Reflective 

Therapy 

1. Do pretherapy characteristics jd 
clients relate differentially to the clients 
reactions to a reflective and to а a 
type of therapy? An answer was men 
in relation to each of the following client 
pretherapy characteristic variables: (o) 
need for autonomy, (b) need for е 
ance, (с) need for deference, (d) need à 
aggression, (е) tolerance-intolerance | % 
cognitive ambiguity, and (f) a 
ness. The relationship of each ol ed 
preceding variables to client reactions 7 
leading and to reflective types of peg 
Was explored with respect to (a) j^ 
therapeutic relationship as vin 05 
clients, and (b) the amount of delen 
verbal behavior exhibited by clients 
therapeutic interviews, ; " 

2. Does clients’ verbal behavior in p 
apy differ in а reflective and a peun 
type of therapy? The client verbal le- 
havior vari (0) s 
pendence, ( aui 
(d) covert 
sistance, 


ables explored were 
b) openness, (c) ats те- 
resistance, and (е) Ove 


cli- 
3. Does the relationship between we 
ent and his therapist differ in a ed 
anda leading type of therapy? The € 


relationship variables consisted of clients’ 
subjective positive and defensive reac- 
tions to therapy and therapist. The ther- 
apist relationship variables consisted of 
the therapists’ subjective positive and 
negative reactions to the clients and the 
therapy situation. An answer was sought 
at the fourth and eighth interviews. 

4. Do changes in clients through ther- 
apy differ in a reflective and a leading 
type of therapy? The client change vari- 
ables investigated were (а) level of mal- 
adjustment, (b) anxiety, (с) defensive- 
ness, (d) dependency, (c) positive atti- 
tudes toward self, (f) positive attitudes 
toward others, and (g) therapists’ evalua- 
Чоп of client changes. 


Effects of Therapists as Individuals 


oo personal characteristics of ther- 
و‎ related to the effects therapists 
on their clients? The therapist per- 
Sonal characteristic variables investigated 
tie (а) ability to enter the phenomeno- 
ae field of another, (b) sympathetic 
Ча] est, (с) acceptance of others, (d) so- 
dize үш value, (е) need to aggran- 
Pul he self, and (f) aggression. The 
is. on clients which these character- 
чен might have were explored with re- 
ae: to (a) clients’ defensive verbal be- 
"Avior in therapy interviews, (b) the 
Client relationship variables, and (с) 
client changes in maladjustment through 
therapy. 
3. Are there differences among thera- 
үа їп the way they affect clients’ verbal 
avior in therapy? The client verbal 
behavior variables explored were (a) de- 
Pendence, (b) openness, (c) guardedness, 
~) Covert resistance, and (е) overt re- 
Sistance, 
3: Are there differences among thera- 
а in the relationship they establish 
their clients? This question was ех- 
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plored with respect to clients’ positive 
and defensive views of the relationship 
and to therapists’ positive and negative 
views of the relationship at both the 
fourth and eighth interviews. 

4. Are there differences among thera- 
pists іп the changes they produce in 
their clients during therapy? The client 
change variables investigated were (а) 
level of maladjustment, (b) anxiety, (с) 
defensiveness, (d) dependency, (е) posi- 
tive attitudes toward self, (f) positive 
attitudes toward others, and (д) thera- 
pists’ evaluation of client changes. 


Effects of the Interaction Between Ther- 
apists and Type of Therapy Admin- 
istered 
1. Is client verbal behavior in therapy 

affected by the interaction of the thera- 

pist as an individual with the type of 
therapy he is employing? The client ver- 
bal behavior variables explored were 

(a) dependence, (b) openness, (c) guard- 

edness, (d) covert resistance, and (е) overt 

resistance. 

ə. Is the therapeutic relationship af- 
fected by the interaction of therapists as 
individuals with the type of therapy be- 
ing employed? This question was ex- 
plored with respect to clients’ positive 
and defensive views of the relationship 
and to therapists’ positive and negative 
views of the relationship at both the 
fourth and eighth interviews. 

3. Is the extent of change in clients 
through therapy affected by the interac- 
tion of therapists as individuals with the 
type of therapy they are employing? The 
client change variables investigated were 
(a) level of maladjustment, (b) anxiety, 
(с) defensiveness, (d) dependency, (е) 
positive attitudes toward self, (f) positive 
attitudes toward others, (g) therapists’ 
evaluation of change scores. 
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EXPERIMENTAL DESIGN AND PROCEDURES 


Independent Variables 


The effects of two independent vari- 
ables were examined in this study. The 
first was the type of therapy administered 
and the second was the therapist as an 
individual. The type of therapy was 
manipulated by defining two families of 
therapist verbal responses. 


Reflective Therapy 


This family of responses included restate- 
ment of content, reflection of feeling, nondirec- 
tive leads, and nondirective structuring re- 
sponses. This therapy was built largely on the 
Rogerian approach (25, 24). Therapists’ be- 
havior was guided by the following working 
hypotheses. 

The therapist attempts to create a warm, 
acceptant, understanding, noncritical psychologi- 
cal atmosphere; to understand and accept the 
feelings which the client experiences as a result 
of his perception; and to communicate this ac- 
ceptance and understanding to the client. 

The therapist believes the client has within 
himself a capacity to understand himself and 
à capacity and tendency to reorganize himself. 
The therapist also believes that, in a warm, ac. 
ceptant, understanding, and noncritical atmos- 
phere, the client will reorganize himself at a rate 
and to a depth most appropriate for him. 

It is necessary for the therapist to accept and 
clarify only those thoughts and feelings which 
the therapist believes are in the client's present 
phenomenological field. These thoughts and 
feelings must be strongly implied by 
himself, if they are not explicitly 
either verbally or nonverbally, 
maintaining this role, the thera 


client to eliminate his need for defenses in the 
therapeutic situation, to recognize his conflicts, 
his emotional reactions and needs, and to bring 
about a self-reorganization of his patterns of 
perception and behavior, 


the client 
communicated 
By consistently 
pist enables the 


Leading Therapy 


The sccond family of responses was 
of directive leads, interpretations, direc 
turing. approval, encouragement, Suggestion, ad. 
vice, information giving, and persuasion, The 
leading therapy was based largely on the ap- 
proaches of Dollard and Miller (6) ang Fromm: 


composed 
tive struc- 


Reichmann (11). It was guided by the following 
working hypotheses. 

The therapist attempts to create a warm, | 
cepting, understanding, noncritical psychologi al 
atmosphere; to contrast the clients report of 
his situation and difficulties with an objective 
reality as the therapist deduces it; to formulate 
hypotheses about the defenses which protect the 
conflicts; and to intervene in such a v that 
he helps the client understand the nature and 
function of the defenses. The therapist may then 
help the client in coping directly with under- 
lying conflicts at a level which the therapist 
deems advisable and feasible within the limi 
tions of time and the client's personal dynamics. 
He thus helps the client to become reoriented 
in terms of reality, 

The therapist believes that the client has а 
capacity to learn new behavior patterns, but 
that this capacity is not being utilized effectively 
because the client's defenses, inappropriate re 
tion patterns, and fears prohibit his becoming 
aware of, and trying out, alternative patterns 
of perception and behavior, 

It is necessarv for the therapist tọ introduce. 
or direct attention to, factors not within the 
client's present awareness, in order to make the 
client aware of his defenses, to help him modify 
them or eliminate the need for them, to recog- 
nize his conflicts, emotional reactions, and needs. 
and to bring the client to adopt alternative 
patterns of perception and behavior. 


Therapists 


Therapists as individuals constituted the sec 
ond independent variable, Ten therapists were 
used in thi Study. It should be noted that the 
authors did not participate as therapists. ү 
therapists were all advanced graduate aon 
ranging in age from 24 tọ 30 years. Six of d 
therapists had internship experience in à ma 
cal setting approximating one year or йб 
One had experience as a mental hospital attent 
ant, caseworker, interviewer, and college Ө; 
selor. One had approximately а уса аре. 
ence with vocational and personal counscling їй 
а university setting. Another had а year of a: 
perience with vocational and personal oo 
Ing In a university setting plus work in a aa a 
for delinquent girls under the supervision @ 
psychiatrist and a social worker. Another i 
worked three months doing casework in à 888 ri- 
ment house, In additi ni 
ence in military, 
They reported th 
still in a formati 
the experiment a 
tion toward a le 


on, some had work EXP Ss 
private, and public Lp 
at their therapeutic biases © i 
ve stage, though at the Ie 
Il but one reported an p nad 
ading type of treatment. Al 
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had supervised experience in vocational and 
educational counseling in the Pennsylvania State 
University Psychological Clinic. 

Prior to the beginning of the experiment. the 
10 therapists had just completed a course in 
psychotherapy. In the course, they studied svs- 
tems representative of both types of therapy. In 
addition, each person had carried. two or more 
therapy cases under the supervision of an ex- 
perienced clinical psychologist. The therapists 
also participated in а training program devised 
especially in preparation for this research. 
Through readings, practice with typescripts of 
previous cases, role playing, and discussions. 
the therapists were familiarized with the re- 
sponse families and given practice in their use. 
On the basis of the final role-playing session. 
all of the therapists were judged to be difer- 
entiating the two types of therapy. 
tem for coding therapists’ responses was 
devised and used to code responses from eight 
recorded interviews for cach therapist. In a 
Pilot study designed to demonstrate the relia- 
bility of the coding system, three of four judges 
coding independently agreed 02%, of the time 
аз to which of the response families a given 
response belonged, ‘Three of four judges agreed 
82%, of the time that a given response belonged 
In one of i5 different categories. The coding 
System used. was patterned after that of a pre- 
“lous research project. (30). A criterion. of ap- 
Proximately two-thirds of each therapis 
perimental responses in the appropriate response 
family for all clients was established as the mini- 
mum acceptable differentiation of treatments. 
In addition, for a therapist to qualify, at least 
60%, of all his experimental responses had to be 
m the appropriate response family for each in- 
dividual client. Six of the 10 therapists met the 
criteria, Only these six therapists and their 24 
clients are included in the principal statistical 
analyses of this study. Some of the qualitative 
Observations made in the study rest on all 10 
therapists and their 40 clients. ble 1 shows the 
distribution of responses in the 15 response cate- 
Bories for all 10 therapists. Clearly, restatement 
Of content and clarification of feeling responses 
Were emphasized in the reflective therapy, while 
Interpretations and directive leads were empha- 
Sized in the leading therapy. If the nonexperi- 
mental categories of XESCFD, XTR, XUN, and 
XUNT (scc Table 1) are excluded and the re- 
Sponses of the six therapists meeting the criteria 
are examined, 89% of their reflective therapy 
responses were appropriate to that therapy 
While 8, 5 of their leading therapy responses 
Were appropriate to that therapy. The nondirec- 
tive emphasis compares favorably with previous 
research describing the verbal pattern of Rog- 


APY 


-I 


TABLE 1 
THE NUMBER AND PROPORTION OF THERAPISTS 
PO Ч SE CATEGORY FOR 
EACH or 1 ү 


| 
Reflectiv * Leading The 

Response —— 

"ate-. | Number %of | Number. % of 

i of Re- Total Re- of Re- Total Re- 

| sponses sponses | sponses spon 


ECS 18 ix | 1 | E. 
XND 16 1.0 | 14 d 
XRC 547 34-4 | 168 | 8.6 
ХСЕ 500 32.1 123 | 0.3 
35 | 25:3 54 2.8 
ao | 9 52 2.7 
36 2.3 | 30 1.8 
51 3.2 59 3.0 
° 4 | 32 1.6 
155 0.8 КА] 45.1 
13 8.2 358 18.3 
39 2.4 | 99 | pi 
11 7 44 2 
3 | «2 32 б 
1 л | 2 E 
a Based on part or all of то therapis re- 


on 40 records for each treatment. 
75 — Nondirective structuring. 
XND—Nondirective leads. 
XRC—Restatement of content. 

XCF— Clarification of feeling. 


XDL— Directive leads. 

XI T— Interpretation. 

XIF—Information giving. 

XAER-— Approval, encouragement, reassur- 
ance. 

DC— Direct criticism. 

SAP— Suggestion, advice, persuasion, 


x 
X 


crian therapy (26, 29, 30). Research on leading 
treatment response-patterns is not available for 
compa son. 


Dependent Variables 
Client Variables 


The major criteria used in selecting 
the client variables were: (a) the measure 
had to have a logical and meaningful 
relationship to the therapeutic process; 
(b) the measure had to be obtainable 
without placing unreasonable demands 
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on the client population; and (c) the 
measure should reasonably be expected 
to demonstrate differences between a re- 
flective and a leading type of psycho- 
therapy. 


Client verbal behavior in therapy. Five main 
variables and one composite variable were based 
on Clients’ verbal behavior in the first four inter- 
views. A tentative classification system based 
in part on previous research (4, 5, 15, 18, 19, 27) 
was devised by the four authors. A pilot study 
on nonexperimental recorded interviews pro- 
duced several modifications of the tentative sys- 
tem. The coding rules were intended to identify 
responses reflecting general sets, rather than re- 
sponses specific to individual therapists state- 
ments. Descriptions of the verbal behavior sum- 
marizing the client verbal behavior variables 
follow. 

1, Dependence: The extent to which the cli- 
ent asks the therapist for his opinions, advice, 
information, evaluation, and instruction, or 


demonstrates a need for structuring from the 
therapist. 


2. Openness: The extent to which the client 
freely discusses his problems, his deviations from 
the “normal,” his culturally frowned-upon traits, 
behavior, and motivations; and, in general, his 
willingness to expose himself to potential criti- 
cism and change; especially his willingness to 
discuss thoroughly those areas which seem most 
threatening. The client does this without at the 
same time qualifying, hedging, and engaging in 
defensive verbal maneuvers, 

3. Guardedness: 


problems 


normal," and denial 
of culturally undesirable feelings, traits, and 


motivations. It also includes the need to justify 
himself or his actions to the therapist, and ex- 
pectations of criticism from the therapist. 

4. Covert Resistance: The extent to which the 
client manifests indirect or impersonalized criti- 
cism of the therapist or therapy. It includes 
blocking, delaying tactics, failure to recall or 
report things, changing the subject, and inter- 
rupting the therapist. It is resistance or hos- 
tility toward therapy, therapist, progress in 
therapy, or toward things which are thought of 
as being conducive to such progress, But the 
resistance is not directly expressed verbally; in- 
stead, other subtle escapes or hostilities are re- 
sorted to by the client. 

5- Overt Resistance: The extent to which the 


cient verbalizes criticism—in an open way— 
the therapist or the therapeutic method. It in- 
cludes personal and verbalized opposition ш 
staying within the limits set by the particular 
kind of therapy which the client is receiving. 
This is verbalized unwillingness as opposed to 
"inability" or failure per se. И 

6. Defensiveness: The sum of guardedness, 
covert resistance, and overt resistance. . 

A time interval was chosen as the unit of 
response so that reliability for cach client те- 
sponse could be determined when the coding 
was done aurally from tapes and discs (all cod- 
ing was conducted in this fashion). The unit of 
measurement, or the client "response" to be 
coded, was a 15-second interval of client verbal 
behavior. Every second of the interview during 
which the therapist was not talking was re- 
garded as consisting of the client's verbal be- 
havior, Each client response was coded in one 
of the experimental categories or in a “none 
category, In this manner, all of the client's 
verbal behavior was classified. A client's score 
оп а given variable was thc percentage of АП 
his responses which were so classified. In the 
opinion of the authors, the aural method of 


coding contained many advantages over type- 
scripts, 


The four authors were the coders in the 
reliability study. In addition to many hours of 
Previous experience with the categor they 
underwent a training program of approximately 
15 hours, Each coder simultancously but in- 
dependently classified client т sponses on four 
nonexperimental recorded interviews which 
Were representative of the two experimental 


therapies. Of the 725 client responses on the 
records, 468 


were classified by three or АП 
judges as falling outside of the experimenta 
Categories, Thirty-five per cent, or 257 TC 
Sponses, were placed under one of the dependent 
Variables by two or more judges. In this com: 
bined task of locating and categorizing ше 
experimenta] responses, at least three out me 
four coders were in agreement on 68%, of п 
responses; at least half of the coders agreed 957 
of the time. When coder agreement among the 
experimental categories alone is considered, €X- 
cluding differentiation of an experimental from 
nonexperimental Tesponse from the data, all e 
ders agreed on 81% of the client response 
Three or more of the coders agreed on 89 fo 
ОЁ the responses, These figures compare p 
ably with reliability studies reported in ome 


eae H H H e 
Investigations (15) in which client responses W 
coded from transcripts, 


ient 
On the records classified for the ехрегіте 
proper, 


the average client made 162 respons 
during the 45-minute interview, Of these. h 
(32%) were Tesponses which fell under one of t 
depend 


T ows 
ent variable classifications, Table 2 sh 


_——— 
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PERCENTAGE ОЕ ALL CLIENT RESPONSE: G 
T VERBAL BEHAVIOR VARIABLES 


Major CLIE 


TABLE: а 


5 FALLING UNDER EACH or THE 


Overt 


es Ж... = Covert 
Dependence Openness Guardedness ВЕ дап Ploctefance 
Reflective Sample 2.26 4.45 3.92 23.58 .63 
Leading Sample 2.67 3-47 6.40 16.00 +72 
А 1 
Combined Sample 2.40 3.96 5.20 19.79 -68 
the percentage of all client responses falling The second scale is intended to reflect posi- 


under cach of the major client verbal behavior 
variables for the clients under the reflective 
therapy, the leading therapy, and these samples 
combined, 

To obtain some information on the validity— 
in the sense of agreement with qualified opinion 
~and objectivity of the experimental coding, a 
Process Rating Scale was devised. This scale 
ained the summary descriptions of the five 
Major client verbal behavior variables. Thera- 
pists were asked to rate on a five-point scale 
the extent to which each of their clients ex- 
hibited dependence, openness, guardedness, overt 
resistance, and covert resistance. In order to 
maintain naivete the therapists were not given 
the scale until the close of the experiment. To 
make their rating more comparable in time to 
the experimental coding, they were asked to 
Tecall the behavior of their clients as it was in 
the first four interviews and rate accordingly. 
With 38 degrees of freedom, + 31 and + 40 
are significant at the .o5 and .o1 levels re- 
Spectively, The correlations of therapists’ rat- 
1185 with the scores obtained through coding 
Were as follows: dependence .55; openness — .23; 
Buardedness .03: overt resistance 59; and covert 
ance .35. Considering the different nature 
of the types of measurement and the memory 
distortion that could have entered the ratings, 
these correlations seem to speak very well for 
the inherent power of the operational definitions 
and the reliability and objectivity of measure- 
ment in the experimental coding of at least 
three of the variables: dependence, overt re- 
Sistance, and covert resistance. 

Relationship: The Client Personal Reaction 
Questionnaire. This questionnaire (called the 
CPRQ hereafter), constructed by the authors, is 
Composed of two 4o-item scales. 

One scale is intended to measure defensive 
Subjective reactions to therapist and therapy. It 
includes items reflecting denial, distortion, with- 
drawal, justification, rationalization, projection, 
hostility, | evasiveness, blocking of thought, 
blocking of speech, obscuring or confusing is- 
Sues, anger, fear, criticism, resentment, and self- 
deprecation. 


tive subjective reactions to therapist and therapy, 
including a sense of progress, achievement, or 
accomplishment; feelings of identification and 
involvement with therapy and/or the therapist: 
feelings of ifene: and/or security in the 
therapy situation; satisfaction of needs for ac- 
ceptance, understanding, help, approval re- 
spect, encouragement; and feelings of respect, 
admiration, confidence, and gratitude toward 
the. therapist, 

The two scales were constructed by having 
six advanced graduate students in clinical psy- 
chology write items to fit definitions for the two 
scales. Each of the items thus obtained was giv- 
en a rating from one to four (poorest to best) 
by each of the four authors. Those items with 
the highest average ratings were included in the 
scale. These final items were further screened 
to avoid overemphasizing one type of reaction. 
Items which involved making judgments about 
therapist or therapy such as, "My therapist is 
well educated," were excluded. Only statements 
likely to clicit a subjective personal reaction 
were included, c.g. Ty therapist is a nice guy." 

Test-retest correlations were computed from 
data obtained in the experiment. Such correla- 
tions have treatment-effects intervening, but are 
sull worth noting. The defensive CPRQ had a 
test-retest correlation of 79 (p < .oo1) The 
positive CPRQ had a test-retest correlation of 
52 (p < .01). The positive and defensive scales 
obtained at two points in therapy correlated 
— -35 (p < .05) at the fourth interview and 
— 31 (p < .05) at the eighth interview. 

The Edwards Personal Preference Schedule. 
This test (7) measures 15 personality variables 
which have their origin in a list of manifest 
needs presented by H. A. Murray and others. 
Five needs from this test were used as client pre- 
therapy characteristics. They were the need for 
deference, autonomy, succorance, 
and aggression. 

Tolerance-Intolerance of Cognitive Ambigui 
Test. This test by Siegel (27) consists of 20 pic- 
tures and 20 unrelated statements. Pictures and 
statements were both rand 


t c omly selected by 
Siegel from different groups of magazines, The 


dominance, 
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client was instructed to compare the ar 
with the pictures, and to indicate if he fe a 
any of the persons pictured made any с Е 
statements. He was to match only those Ag ie 
were appropriate. The greater the fendency qp 
associate statements with pictures, the lower was 
ance for ambiguity. 

„о Problem Check List. The meas- 
ures used from this test (20) were the total 
number of problems checked and the number 
of words used by the client in summarizing his 
problem. These two measures had served as 
measures of client defensiveness in a previous 
study (30). The assumption was made that the 
more restricted a client was in 
discussing problems, the higher was the level of 
defensiveness. 

The Minnesota Multiphasic Personality In- 
ventory. Six scales derived from the population 
of items on this inventory were used in the 
present study: Maladjustment Index (12), Taylor 
Anxiety Scale (32), Defensiveness Scale (13), De- 
pendency Scale (22), Positive Attitude ‘Toward 
Self Scale (14). апа Positive Attitude Toward 
Others Scale (14). These variables were intended 
to reflect the pre- to post-therapy changes, 

The Therapist Posttherapy Rating Scale. 
"This scale, developed by the present research 
group, was modeled after an carlier scale (33). 
It consists of s7 reflect. changes 

ts undergoing therapy, 
ected by judges from a population 
of 77 items as being those most likely to reflect 
changes resulting from therapy, 


admitting and 


Therapist Variables 


The therapist-characteristic у 
chosen for Study were 
eral criteria in mind: 
enduring nature, ie., 
ableness as 


ariables 
selected with sey- 
(a) their relatively 
seeming unalter- 
a result of specific training; 
(b) their logical relationship to impor- 
tant aspects of prescribed t 


herapist be- 
havior; (c) their measurability; (d) their 
being relatively unbiased by the psycho- 
logical sophistication of the therapists; 
(е) their objectivity of scoring, 


Ability to Enter the Phenomenolog 
of Another. This was defined as inte 
ing about the internal frames of 
others and being able to see how o 
and feel in terms of these interna] frames of 
reference, The Intraception scale from the Ed- 
wards Personal Preference Schedule, and a 


ical Field 
rest in learn- 
reference of 
thers perceive 


measure of role-playing ability devised by. the 
authors were used to measure this variable. The 
latter measure was constructed by asking thera 
pists to play the role of a client they know pik 
while a cotherapist plaved the role of a beading 
‘Treatment therapist. The therapists understooc 
this to be part of the training program. Bill: 
ings of the realism of the role played were : 
signed for three separate. dimensions of rales 
playing ability. ‘These dimensions were (a) thc 
content, i. what the "client" talked about. (b) 
the reactions that the "client role-player’ showed 
to the therapist's leads, (c) the affect which. the 
role-player displayed while acting as e 
Four judges assigned the ratings independent у 
after practice in learning to make reliable judg- 
ments. A scoring guide was prepared darmi 
five points for cach dimension, Complete agree 
ment was obtained among the four judges on 


65° of the judgments, while 94%% of the judg- 


o j 
ments were either in complete agreement or only 
опе step removed from the consensus. as d 
Sympathetic Interest, This was defined as 
kindly interest in the activities and thoughts E: 
others and was measured by therapists! scores 
on the Edwards Nurturance scale, А as 
Acceptance of Others. This was defined iei 
the willingness and ‘or ability of the цат 
to understand and accept what the client A 
to say without fecling a need to evaluate, jut ВЕ 
or make criticism cither openly, or in his ow a 
mind. The authors Constructed a test са led a 
Test of Clinica] Judgment to measure E 
able. It consists of items purported to be sta 


4 ade bv 
ments of beliefs, opinions, and values made b? 
unidentified 


individuals. These items repre r^ 
viewpoints deviant from those of this ipeo 
general and those of psychologists in pantie 
However, the items do not cy press viewpoints $ Р 
deviant as to Warrant their ne arily being on 
ceived of as pathological, ‘The p an 
to classify the st 
adjusted or mal 
soned that 
maladjusted 


apisis were y 
atements as being made D» ea- 
adjusted individual. It want as 
those who classified fewer iene 
Would be those who were most uly 
cepting of the values of others. A pilot peni 
With 30 clinica] Psychology graduate pe to 
Supported the idea that the items intende " 
be ambiguous could be viewed as 

or “maladjusted,” Six clearly pathological 1 a 
six clearly normal statements were шс а 
filler items to provide a frame of reference The 
as a kind of Validity check, Twenty-eight of 


В А oup 
thirty persons taking the test in the pilot g "i 
classified the filler items as intended by ” 
authors 


? fed 
» indicating that items clearly “adjust 
or “maladjusted” could be so classified. 1 
Social Stimulus Value, This was define on 
the favorable effect the individual produces so- 
others with whom he has social contact. 2 


as 


— 


ciometric measure devised by the authors was 
used for this variable, Therapists were asked to 
select the two most preferred and two least pre- 
ferred members of their group in five different 
Social situations which involved confiding threat- 
ening criticisms, personal friendship, cooperative 
Work, professional and personal 
therapy, Scores were derived reflecting the de- 
Bree of preference for cach of the therapists. 

Need for Agerandizing the Self. This was de- 
fined as the need to make oneself important. by 
gaining the attention, admiration, and awe of 
others, Therapists’ Edwards" Ex- 
hibitionism scale were used to measure this need. 

Aggressiveness, This was measured by the 

Aggression seale from the Edwards. 
Р Relationship: The Therapist Personal. Reac- 
tion Questionnaire, This questionnaire (called 
the TPRO hereafter), constructed by the au- 
thors, is composed of two scales of 35 items cach. 
One scale is intended to reflect Negative Re- 
Actions to therapy and client, and includes items 
reflecting feelings of hostility, resentment, criti- 
SSM, superiority toward the client: feelings of 
doubt, discouragement, uncertainty, and failure 
In regard to progress and accomplishment with 
the client in therapy: feelings of anxiety, dis- 
Pleasure, discomfort, boredom in anticipation of 
Or in the interviews: feelings of incompetence, 
adequacy, ineffectiveness, lack of understand 
Ng, and inability to help both in regard to in- 
'erview behavior and in the long тип: feeling 
disliked, rejected, ridiculed, and pushed, 

The Positive Scale reflects feelings of progress. 
Achievement, and accomplishment with the 
Client in therapy; feclings of identification and 
Involvement with the client; feclings of comfort, 
Pleasure, and anticipation in relationship to the 
interview hour; feelings of respect, admiration, 
Sympathy, and affection for the client; and gra- 
tification of existing needs such as those for ap- 
Proval, respect, and therapeutic competence, 

The construction of the TPRQ was identical 
19 that of the Client PRQ previously described. 
Гез тес correlations were obtained by corre- 
ating а score obtained at the fourth interview 
It à Store obtained at the eighth interview. 


supervision, 


scores оп the 


The Negative scale had a test-retest correlation 
of 85 (p< 001), while the positive scale had a 
rest-retest correlation of .&1 (p < ооз). The 
Positive and negative scales correlated 
(> a0) at the fourth interview and 
(p 5 10) at the cighth interview. 
Clients 
Most of the clients used in this study 
Wer s А : 
2 young adults in their twenties 
Who; К x d ERR 
lose Symptoms were primarily neurotic 
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in character. Each therapist had two 
clients in each treatment. Thus, there 
were 24 clients for the six therapists re- 
ported in this study. Seven of the clients 
17 were men. There 
were two women in the reflective 


were women and 
and 
five in the leading therapy. The present- 
ing symptoms often included some refer- 
ence to unsatisfactory academic perform- 
ance, since the sample consisted largely 
of university students. As therapy pro- 
gressed, however, it was usually apparent 
that the academic problem was primarily 
Problems included in- 
ability to get along with peers or parents, 


symptomatic. 


feclings of inadequacy, sexual frigidity, 
impulses, unsatisfactory 
marital relationships, and disturbing 


homosexual 


emotions such as anxiety and depression, 
The intensity of problems ranged from 
mild to severe, The experiment was ter- 
minated at the end of the spring semes- 
ter because many of the clients were leav- 
ing for the summer. The number of in- 
terviews completed at termination 
ranged from one to 23, with an average 
of 12.8. The fact that a considerable 
proportion of the clients continued their 
interviews after the experiment was ter- 
minated indicated that they had not 
completed their therapy at that point. 
The sample of clients described includes 
those who completed four or more inter- 
views. (In the part of this study con- 
cerned with client verbal behavior in 
therapy, there was no criterion of four 
interviews. The first four clients com- 
pleting one or more interviews for each 
therapist were included. However, this 
sample differed from the former by only 
two clients so that the samples are almost 
identical.) 


Procedures 


Clients were randomly assigned to 
) 8 
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therapists and to the two therapies. 
Therapists met their clients in 45- to 60- 
minute interviews twice a week on non- 
consecutive days. All interviews were ré 
corded on discs or tapes. Before the ini- 
tial interview, clients completed the 
MMPI, Edward's Personal Preference 
Record, the Mooney Problem Check 
List, and the Tolerance-Intolerance of 
Cognitive Ambiguity Test. After the 
fourth, eighth, fifteenth, and terminal 
interviews, clients and therapists com- 
pleted Personal Reaction Question- 
naires. After the client terminated ther- 

apy or the experiment was terminated, 

whichever occurred first, the client com- 

pleted the MMPI again. In addition, sey- 

eral tests were completed by therapists 

early in the experiment, 
Clients were never told they were participat- 
i - They were informed that 


Statistical Design 


A double Classification analysis of vari. 
ance design was used in the present ex. 
periment. This design made it possible 
to evaluate differences between therapies, 
differences among therapists, and 


differ- 
ences resulting from interaction effects 
of the two independent Variables оп the 


dependent variables, Correlationg be- 
tween pretherapy measures of clients and 
the dependent variables were not large 
enough to warrant analysis of covariance, 

In the case of the client verbal be. 
havior variables, a client’s Score on a 


given variable was the proportion of all 
of his responses which fell under that 
classification. The proportions were 
transformed to angles for the analyses of 
variance (28, р: 316). 

Correlation procedures were used to 
relate client pretherapy personality char- 
acteristics to client reactions during ther- 
apy. Similar procedures were used to re- 
late clients' reactions in therapy to thera- 
pist personality characteristics. A р < ‘05 
Was used as an acceptable level of sig- 
nificance, 


RESULTS AND Discussion 


Client Pretherapy Characteristics 

Tables 3 4, and 5 report pairs of cor- 
relations between client pretherapy char- 
acteristics and three client therapy meas- 
ures, 


These correlations Suggest that the more ag 
gressive a client, the more verbal defens 
Will be manifested in the leading therapy. : 
though the correlation in the reflective thera 
does not meet the criterion for statistical SEL 
nificance by itself, it is large enough to SURES 
the Possibility that the more aggressive ЫК: 
the less verba] defensiveness will be manifestet 
in the reflective therapy (Table 3). eir 

Clients who Were less willing to discuss ed 
problems on tests prior to therapy also tend in 
be more defensive. in their verbal behavior pa 
the leading therapy, while clients with less P? 


: с ofensive 
therapy defensiveness tended to be less defen 
verbally in the | 


to elaborate alc 
not to re] 
in the refi 

Deferen 
defensive reactions, 


efensive reactions in the leat 
© seemed to be no consi S 
en the trait of deference d a 
€ reactions of clients towar 
reflective therapy (Table 5), vith 
ients tended to react ding 
defensive reactions to lea ted 
hereas less autonomous clients de to 
e defensive reactions, There Se EAD 
€ nO consistent relationship between this ару 
and clients’ reactions to the reflective ther 
(Table 5) 


Ё . prela- 
It is worth noting that three cor! 


n 


| 
| 


EFFECTS OF A REFLECTIVE AND A LFADING TYPE OF PSYCHOTHERAPY 13 


TABLE 3 


CLIENT PRETHE 
ENSIVE? 


APY MEASURES AND THE CLIENT 
Ess SCALE 


CORRELATIONS BETWE 
VERBAL BEHAVIOR DE 


adi xi TN Significance 
Client Pretherapy Measures Беш кен Ai of Difference Р 
шы SEA between z's" 
TICA» — 36 = d >. 
Deference —.12 .o9 l1. 
Autonomy н =й on S. 
Succorance pa me ты E 
Dominance eon =} S: 
Aggression .42 = 37 ду zh 
Mooney: Number of problems —.28 —.10 é Б 
Mooney; Number of words — .60° лт Bu Ж, 


^ Based on Fisher's s. 


Н l'olerance-IEntolerance of Cognitive Ambiguity. 
77.444 required to be significant at .os level. 


TABLE 4 
iRAPY MEASURES AND CLIENT POSITIVE PERSONAL 


CORRELATIONS BETWEEN CLIENT PRET 


REACTION QUESTIONNAIRE AT THE FOURTH [NTERVIEW 
E Р ы Significance 
i „ег Reflective САН 
Client Pretherapy Measures Made Thera ae of Differences p 
erap} ару Between r's* 
a 
| СА .38 —.os 1.31 > .05 
eference —.1] ‚21 TT >.05 
utonomy —.04 سه‎ 1.25 >.05 
| Puccorance 32 .от .93 2.05 
Р ominance —.24 Кр] .96 2.05 
p gression —.1$ —.2$ .32 >.о$ 
‘looney Number of problems +22 —.27 1.46 2.05 
Оопеу: Number of words „17 —.05 -64 >.о$ 
^ Based on Fisher's =. 
TABLE 5 


CORRELATIONS BETWEEN VARIOUS CLIENT PRETHERAPY MEASURES AND THE CLIENT DEFENSIVE 
PERSONAL REACTION QUESTIONNAIRE, FROM THE FOURTH INTERVIEW 


" Ы Significance 
; Leadin; Reflective ; 

Client Pretherapy Measures Therapy Therapy тшеп Р 

TICA .29 —.09 I.I14 > .05 
Deference ‚46° — 28 2.30 <.05 
S utonomy —.$2° .18 2.22 <.05 
Uccorance —.09 E К; > .о5 
Ominance —.04 —.24 -58 >.о$ 
Meression —.09 .o8 +50 >.05 
Coney: Number of problems — ;07 -35 1.28 > .05 

Coney: Number of words —.22 +23 1.31 >.о$ 


^ Based on Fisher's s. " 
7 7.444 required to be significant at .os level. 
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tions for the leading treatment sample 
were statistically significant while there 
were no significant correlations for the 
reflective treatment sample. This sug- 
gests that certain client reactions, at least 
during the first four interviews, are more 
predictable in a leading than in a reflec- 
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Differences Between Therapies 
Twenty-one analyses of variance were 
completed. Table 6 reports these data. 
Only 2 of the 21 showed a statistically 
significant difference between the two 
therapies. The similarity of the effects 
produced on clients by the two therapies 


TABLE 6 


F Ratios AND PROBABILITY STATEME 
ON WHICH ANALYS 


Reflective 


Variable Mean Eo F Ratio b 
Client Nerbal Behavior Variables: 
Dependence 2.26 "E 
Guardedness Я e 2.67 .26 > .20 
Openness avs 6.49 4.02 <.05>.01 
Covert Resistance 23.58 3-47 106 2-29 
Overt Resistance 3 б, 10.00 +02 2-20 
Defensiveness P “72 +20 pr 20. 
Relationship Variables: 22.6 21.0 .05 2.20 
Defensive CPRQ—gth 2 ` 
Defensive CPRO—8th 200.5 213.1 > 20 
Positive CPRQ—ath are 197.3 1 <.20> .10 
Positive СРКО sth ae 333.0 > .20 
Negative TPRQ—ath 317.3 342.9 2 <i, 20.> 10 
Negative TPRQ—8th 72.8 67.0 > .20 
Positive TPRQ—gth 74.8 01.3 1 <.20>.10 
Positive TPRO—8th peni 97.3 5.2 
Change Variables: 99.1 105.1 >.20 
Maladjustment Index фе 
Jependency 25 —6 
Belensiv ness +1.90 د‎ id > a 
Positive Attitudes Toward Self То. 5o +оо; x S20 
Positive Attitude "оммага Others To.35 T 73 of о> © 
Taylor Anxiety Scale 7 То.7о +о: 55 е ч "> .20 
I terapist Posttherapy Rating Scale p 7245 1 $ > ee 
T Bees "-—— "i 5 91.f 1 ss 
_® an analyses yere made on the желу | — 6.32 <.0$ 
ay*or and Posttherapy Rating Sc. 1. P € " between pre- ; the 
cates a decrease in the trait mese cae where Posttherapy s, and Post-therapy scores, except for, adi- 
^ A higher score on the Th 80, TRIS. Were 


К nerapist р, 
greater improvement. Pist Postthe 


tive therapy. It is interestin 
that the four signific 
correlation involved 
defensiveness, one 


5 to note also 


ant differences in 
measures of client 
a behavioral me 
ance 
measure of the client’s Subjective тег 

tions toward therapy. This Suggests ea 
defensiveness in a leading therapy + A 
characteristic which is predict; 
client pretherapy characteristic 


asure 


of in-therapy resist and the other 
a 


$. 


тару Rating Scale 


used. А decrease in score 


T Е ent 
Indicates a therapist judgme? 


ы. | 
15 a prominei - 


t finding in this area 
the $ гару 
Da Study, On the "Therapist Postcheral j 
Rating Scale ently 
rated clients f 
show 


therapists consist 
» " ару 
; її the leading фер), 
Ing more improvement than clic 


i $ , ng 
hs the reflective therapy, This ne 
nus " xg An 

USt be accepted with qualifica" е 
however oh 


E nist € 
> Since all but one therap!? e 

á E : p 

а preference for a leading tY Р jecit 
Prior to the experiment. 


presseq 
therapy 
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preferences may have affected their rat- 
ings. Clients’ verbal behavior during the 
first four interviews was more guarded 
in the leading than in the reflective 
therapy. This suggests the leading ther- 
ару may have been somewhat more 
threatening. Guardedness will be con- 
discussion of 


il in the 


sidered in det 
differences among therapists. 
the 
were not impressive, the consistency with 
which trends appeared the 
fourth and cighth interview on the rela- 


Although levels of significance 


between 


tionship measures is worth noting. The 
CPRQ suggests that clients tend to. be- 
defensive. in the reflective 
therapy and more positive in the leading 
therapy. Therapists tended to. become 
somewhat more negative in their reac- 
tions the 


come more 


toward clients in reflective 
therapy, 

Although statistically significant dif- 
ferences between the therapies were few, 
Some trends in the data suggest that the 
two therapies may have somewhat diller- 
ent eflects, Clients in leading therapy 
tended to become less defensive while cli- 
ents 
come 


in reflective therapy tended to be- 
more defensive in their subjective 
reactions toward therapy. Leading ther- 
ару clients tended to show greater posi- 
tive change in their attitudes toward 
themselves, ‘Therapists tended to become 
More negative in their reactions toward 
clients in reflective therapy. Therapists 
Were able to hold clients better in the 
leading therapy than їп the reflective 
therapy, and rated clients in leading ther- 
ару as more improved. On the other 
hand, clients in the reflective therapy 
Were less guarded and tended to exhibit 
less dependence and overt resistance, and 
More openness than clients in leading 
therapy, Covert resistance tended to be 
Sréater in the reflective therapy, due 
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largely to a much greater number ol cli- 
ent long pauses. Some other components 
of covert resistance, namely blocking and 
interruption of the therapist, were no- 
ticeably less frequent. in the reflective 
therapy than in the leading therapy. 


The writers had opportunities to ob- 


serve some clients after they completed 
therapy. These observations indicated 
that improvements were made by clients 
in both types of therapy. 

Several possible explanations may ac- 
count for the fact that there were few 
statistically significant differences 
tween the therapies. The findings may 
mean that the leading and reflective 
therapies do not produce very different 
results. It is also possible that the two 


be- 


do produce ditlerent results but that dil- 
ferences did not show up because of the 
limited. power of the experiment. The 
have been inade- 


measures used 


quate, or it may be that the variables 


may 


themselves were not appropriate to dem- 
onstrate dillerences which actually exist. 


Differences Among "Therapists 


Statistically significant differences 
among therapists appeared in 4 of the 
21 analyses, as shown in Table 7. Three 
of the four differences appeared on the 
relationship variables. Therapists appear 
to produce differing degrees of subjective 
defensive reactions in their clients by 
the fourth interview, but these differ- 
ences tend to disappear by the eighth in- 
terview. On the other hand, differences 
in the degree of positive subjective reac- 
tions elicited from clients are not statis- 
tically significant until the eighth inter- 
view. From the clients’ point of view, 
the defensive aspects of the relationship 
appear to develop earlier in therapy 
than do the positive aspects. Therapists 
differ in their negative reactions to their 
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TABLE 7 


F Ratios AND PROBABILITY STATEMENTS FOR AMONG THERAPISTS EFFECTS ON 21 VARIABLES 
i й FOR WHICH ANALYSES OF VARIANCE WERE COMPLETED 


Variable F Ratio р 
Client Verbal Behavior Variables: " "m 
Dependence © 
Guardedness 7 68 <.01>.001 
Openness 1.08 2.20 
Covert Resistance .86 2.20 
Overt Resistance +71 2.20 
Defensiveness .97 >..20 
Relationship Variables: 
Defensive CPRQ—4th interview 4.22 <.05>.01 
Defensive CPRQ—S8th interview 2.67 <.10> .05 
Positive CPRQ—4th interview 2.08 <.20>.10 
Positive CPRQ—S8th interview 6.88 <.01>.001 
Negative TPROQ—gth interview 3.71 <.05>.01 
Negative TPRQ—8th interview 1.86 <.20>.10 
Positive TPRQ—4th interview 1.55 > .20 
Positive TPRQ—8th interview 2.58 <.10>.05 
Change Variables: 
Maladjustment Index 1.17 > .20 
Dependency 1.31 > .20 
Defensiveness .92 > .20 
Positive Attitudes Toward Self .26 .20 
Positive Attitudes Toward Others 1.67 > .20 
Taylor Anxiety 1.57 >.20 
Therapist Posttherapy Rating Scale .39 2.20 


clients by the fourth interview but these 
differences tend to disappear as therapy 
progresses. Although not statistically sig- 
nificant, the data suggest that differences 
in therapists’ positive reactions tend to 
develop as therapy progresses. From the 
therapists’ point of view, the negative as- 
pects of the relationship appear to de- 
velop more quickly than the positive as- 
pects. Therapists also appear to differ 
among themselves in the amount of 
guarded verbal behavior they elicit from 
their clients during early interviews, 
There were по significant differences 
among these therapists in the changes 
produced in their clients on the client 
change variables investigated. 

For ascertaining the relationships be- 
tween therapists’ personal characteristics 
and client reactions, Pearson product- 
moment correlations were computed be- 
tween all the therapist variables and 
five of the client variables. The client 
variables were the fourth interview Posi. 


tive and Defensive CPRQ, Guarded and 
Defensive verbal behavior in therapy: 
and the Maladjustment change variable. 
The values used for computing these 
correlations were the scores obtained by 
cach therapist on each of the therapist 
characteristic variables and the mean 
scores of his four clients on the client 
variables. The correlations were conr 
puted using the total sample of 10 thera- 
pists. In those instances in which the cor- 


relation coefficients obtained were .30 OF 
above, correl 


ations were computed also 
for the sample of 6 therapists. None of 
the correlations met the criterion for sta- 
tistical significance. This is not surpris- 
ing since the extremely small samples 
provide very limited degrees of freedom 
and thus require a correlation of .63 t° 
be significant at the og level. Howeve^ 
six of the eight correlations compute 

using Edwards’ Nurturance Scale yield 
correlations of .до or above, suggesting 
that the degree to which it is possesset 


سے 


| 
| 
| 
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as a quality by the therapist may have 
some real relationship to his clients’ re- 
actions in therapy. Similarly, the Test of 
Clinical Judgment, intended to reflect 
acceptance of the values of others, con- 
sistently correlated in the expected direc- 
tion, although in magnitude the correla- 
tions were not statistically significant. 
The experimenters feel that the main 
problem behind the failure to find sig- 
nificant relationships between therapist 
characteristics and client reactions lies 
in the approach taken. It is felt that the 
measures of therapist extratherapy be- 
havior which were obtained were too far 
removed from their behavior in therapy. 
The writers now believe that a more 
profitable approach could be made by in- 
vestigating therapists’ behavior in the in- 
terview itself. At this stage of our knowl- 
edge it seems rather futile to continue 
with the investigation of "permanent" 
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traits of therapists in the hope that they 
will relate to client behavior. 

While none of the therapist personal 
characteristics studied related to differ- 
ential reactions of clients, certain pat- 
terns of therapist interview behavior 
seemed to relate qualitatively to their 
clients’ measured behavior. In the course 
of the investigation the experimenters 
listened. to scores of interviews. From 
these interviews, the writers agreed that 
therapists could be grouped along the di- 
mensions of perceptiveness of client dy- 
namics, threat to clients, and warmth 
and friendliness. It should be made clear, 
however, that these groupings were rela- 
tive only to the 10 therapists їп this 
experiment. Characteristics attributed to 
cach group are meant only to depict the 
principal ways in which members of the 
group differed from other therapists in 
the study, Table 8 presents data on se- 


TABLE 8 
MEAN Scores FOR THE Four CLIENTS or EACH THERAPIST ON SELECTED VARIABLES 


GROUPED ACCORDING TO THE TYPE OF THERAPIST 


Conversational T ——! Friendly Dynamic 
Therapists Threatening Therapists Therapists 
A B Є р Е Е G H I T 
Guardedness Ej 1.8 1.9 4.9 6.7 4.1 7-9 Ong Wavy 2.5 
Openness 1.6 2-43 4.8 0.3 3.4 3.7 7.9 9.9 б.о 4.9 
Maladjustment t5.8 10.3 14.5 К; 5.5 15-8 4.0 | —5.3 2.5 —6.0 
Anxiety 21.5 22.5 25.3 25.0 24.8 25.5 24.3 21.8 22.8 26.0 
Dependence* 6.3 TiS 2.0 | —5.5 4.0 3:0 -—I1.5 2.0 2.0 —1.$ 
Defensiveness* 2.0 —.3 —5.0 | —2.0 —1.0 —4.0 о | —r. — po 
Attitudes toward 3 2:8 a 
о{һегв* 1.0 1.0 2.0 ° 2.0 225 1.0 -.5 = 
Attitudes toward S Е ш. 3 
self 3.0 3 1.5 —.5 —1.0 ї.8 5 — 

aU . : : : 8 E =, ees) 
Positive CPRQy | 392.0 303.0 413.0 | 275.0 390.0 376.0 386.0 317.0 MA 374.0 
Positive СРКО | 301.0 343.0 419.0 | 266.0 393.0 308.0 384.0 | 356.0 279.0 346.0 
Defensive СРКО? 186.0 246.0 172.0 | 229.0 202.0 209.0 203.0 | 224.0 170.0 206.0 
Defensive CPRO.” 195.0 208.0 185.0 | 243.0 211.0 178.0 204.0 | 208.0 170.0 212.0 
Positive ТРКО | 107.0 71.0 03.0 | ror.o  9o.o 87.0 108.0 | 122.0 95.0 100.0 
Positive TPRQ | 114.0 60.0 о.о) 98.0 96.0 92.0 06.0 Peco: Geen = 
Negative TPRQ.? 79-0 35.0 62.0] 58.0 70.0 46.0 78.0] 81.0 92.0 75.0 
Negative TPRQs | 74.0 36.0 68.0 | 65.0 63.0 48.0 77-01 gia Bao Шо 


^ Scores represent the difference between pre- and post-therapy 


Provement. 
^ Fourth interview. 
* Eighth interview. 


' scores. Plus scores represent im- 
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lected variables for the three groups of 
therapists discussed below. 


ree therapists were characterized as friend- 
Three i dynamic, and were 
ү. nthreatening, and nondynz А 
у. шй i " therapists. They ap- 
labeled “conversational ther pis y E 
s d to take the role of "friend to the client. 
ed herapists tried to be as nonthreatening 
рр ee i mmunicate to the 
s possible. They tried to со à 
Re бу his side. Their therapy 
lient that they were on his side. у 
E 151$ inly of restating content 
seemed to consist mainly o з 
in Reflective Therapy or being reassuring and 
supportive in Leading Therapy. There was по 
consistent plan for encouraging clients to dis- 
cuss problem arcas. In fact, “conversational 
therapists seemed actively to avoid problem 
areas at times. One need apparently controlling 
their behavior was a strong need for approval 
and acceptance. Clients of these th 
less guarded in their interview verbal behavior, 
Consistently, they showed д decrease on the 
Maladjustment Index of the MMPI from pre- 
to post-therapy. Strong positive reactions toward 
the therapy and therapist were reported on the 
CPRQ, as well as generally low defensiveness 
reactions, 
Four therapists were character 
friendly, threatening, and dy 
These were labeled “threatening” therapists, 
They tended to correct the unrealistic percep- 
tions and plans of the client, Through the 
choice of words, tone, or inflection of voice, 
client thoughts and actions were directly or in. 
directly evaluated. These therapists seemed to 
: К р 
imply that they “knew the correct answers” to 
the client's problems or soon would—that they 
were authorities, They scemed less concerned, 


or less aware, than the other therapists with 
the amount of threat involved in their 


ments, Their interpretations were 
quite extreme and w 


seemed to direct the 
to another impulsively, i 


crapists were 


d as non- 
‘namically oriented, 


State- 
sometimes 
ithout sound basis, They 


” therapists 
and challengia 

apparently con. 
а need to dominate 


Three of the four 
ot maintain adequ 


than other therapists, One need 
trolling their behavior was 
and control the situation, 
therapists who did n 


ate dif- 
ferentiation of the therapies were in this group, 
Scores for clients of “threatening” therapists 
were inconsistent from therapist to therapist, 


However, it is interesting to note that the 
therapist whose clients were the most defensive, 
and least positive, on the relationship measures, 
and least open in the interview, was one in the 
“threatening” group 

The remaining three therapists we 
acterized as warm, skillful, and 
ented: These Were labeled “ Я dynamic» 
therapists. While very warm, led tó ds 
always mindful of the busine l. In the 


TC chay. 
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reflective therapy, they clarified and "peg 
therapeutically relevant feclings or content asa 
out distorting the client's emphasis. In HE ses 
ing therapy they consistently offered leads m P 
pertinent to the client's dynamics and Glaser 19 
the client's. capacity for dealing with the а 
lem. Although all 10 therapists were а d 
adequate, the “warm dynamic” thera oon c 

to approximate morc closely the "ideal {ны 
pist as he is frequently described in the een 
ture. The personal needs of these therapists 
were less obvious and scemed to interfere А 
with the Progress of therapy than the = с 
other therapists, They tended to report jm 
more positive and morc negative personal ie 
actions than the other therapists, The EP 
menters fully expected that clients of 355 
dynamic" therapists would earn the most ME 
sirable scores on the various measures. How- 
ever, in virtually every instance their ФЫЙ 
failed to do so. Moreover, on some variables 


N sirable 
they consistently earned the least desira 
Scores, 


It is difficult to account for the арыш 
poor showing of the clients of the "warm, s A 
namic” therapists, Why did their clients paan 
ently fail to respond in a manner which wou $ 
reflect the warmth, acceptance, and ill е 
tended to them? Similarly, it is difficult to e 
count for the apparent superiority of the к 
of clients of the “conversational” therd pists. [т 
only reasonable explanation seems to be that un 
variables concerned with client reactions b 
therapy ате actually of a different nature Ши 
Was originally conceived, It was expected, tar 
example, that the absence of guarded behavior 
in the interviews would be indicative of a ms 
relationship and greater progress. Noting; 1 ri 
the therapists who appeared most skilled seem re 
to have clients who indulged in relatively Seal 
guarded behavior than other clients, a HE 
Concept of Buardcdness emerged. It was fount 
that while clients of the “warm, A 
therapists were quite guarded, they were al т 
quite open. In fact. the two kinds of beha in- 
seemed to 89 together, The confession of ап a 
adequacy, or a socially unacceptable feeling. ate 
usually preceded or followed by guarded state 
ments. It seemed that clients regularly wc as 
enced some anxiety in relation to such GRENEN 
about them d needed to cling 10 gan 
defense in order to allay their anxiety. E 
versely, clients or “conversational” mec 
showed little tendency to discuss their pus. 
Openly. Thus, they had Tittle to be апо 
about, The relationship between gume e 
and Openness Seemed to reflect what was 0© 


selves an 


1 

and o ; iha! 
ning in therapy far more meaningfully 

either variable Considered. alone. Ja- 

sd 16 : 

The pattern of client scores in 


50 
sor alse 
Поп to therapist interview behavior 
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seemed to help clarify the concepts of 
positive and defensive reactions on the 
relationship measures. Originally, it was 
thought that relatively high defensive 
reactions on the CPRQ would be present 
only in a threatening therapy. Delensive- 
ness was thought to be detrimental to 
therapy. It is now felt that a certain 
amount of defensiveness is probably a 
necessary concomitant of problem solv- 
ing arising from the internal resistance 
of the client to a more realistic evalua- 
tion of himsell. Relatively high positive 
scores on the CPRQ were originally con- 
ceived of as instrumental to therapy. It 
is now believed that high positive scores 
may represent in part dependence on the 
therapist as an authority figure. This 
view arose from the fact that therapists 
assuming benevolent or threatening au- 
thority roles achieved the highest scores 
on the CPRQ at both the fourth and 
eighth interviews. 

The change in maladjustment favored 


clients of the “conversational” therapists 
However, the meaning of this change is 
not clear. From these data, it might ap- 
pear that a highly positive relationship 
is the crucial condition for clients to 
improve in therapy. However, the writers 
Suspect that the nonthreatening atmos- 
phere created by the conversational ther- 
apists resulted in a temporary lessening 
of anxiety which produced changes in 
maladjustment scores. It seems more like- 
ly that a positive relationship is a neces- 
sary but not sufficient. condition for 
effective therapy. As a result of consis- 
tent attention to dynamics in a friendly 
atmosphere, it scems reasonable to ex- 
pect clients of friendly, dynamic thera- 
pists to become more aware of their prob- 
lems and disturbing feelings. The in- 
crease in maladjustment scores at the 
time the experiment was terminated for 
Such clients may reflect this incrcased 
awareness, If effective therapy continued, 


this trend. might be expected to reverse 
itself and a decrease in maladjustment 
scores appear by the end of therapy. 

The fact that the friendly, dynamic 
therapists reported more positive and 
negative reactions may reflect a higher 
degree of sensitivity to, and involvement 
with, their clients. These therapists were 
apparently more aware of their own re- 
actions and consequently were able to 
control their own behavior more effec- 
tively during therapy. 

On the basis of the scores obtained by 
the three types of therapists described, 
it is suggested that a more significant 
index of the effectiveness of therapy may 
be the relationship between guarded and 
open behavior. Similarly the relationship 
between clients’ positive and defensive 
subjective reactions may be an index to a 
truly therapeutic relationship. If there is 
100 much guardedness in relation to 
openness and/or too much defensiveness 
in relation. to positiveness, therapy is 
probably not proceeding optimally. Low 
scores on guardedness with little openness 
may indicate that little problem solving is 
occurring. High scores on guardedness 
with little openness may indicate that 
the therapist is unduly threatening. High 
scores on positiveness with little defen- 
siveness suggest that the relationship may 
be satisfying to the client but not neces- 
sarily therapeutic. It should be pointed 
out that these proposed relationships 
apply only to the carly stages of therapy. 
In later stages the optimal relationships 
of these scores may be quite different, 


Differences Resulting from Interaction 
of Therapist with Therapy 

Three of the 21 analyses revealed 
Statistically significant | interaction 22 
ratios as indicated in Table 9. These 
differences appear in the analyses of the 
client relationship measures, The per- 
sonal qualities which therapists invested 
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E 9 


HOD) EFFECTS ON 


VERE COMPL 


P Rat? b 
Variable F Ratio p 
Client Verbal Behavior Variables: ыб So 
Dependence Ri ss йз шй 
Guardedness 38 Же 
Openness . in eer 
Covert Resistance "9 А 
Overt Resistance -64 poe 
phu m 1.15 * 
ionship Variables: g 
еа CPRQ-—Ath interview 7.92 oi = = 
Defensive CPRQ—8th interview 6.61 <.o1 xr 
Positive CPRQ—4th interview 2.98 «10> .05 
Positive CPRO—8th interview 9.56 = 5507 
Negative TPRQ—ath interview -99 > 330 
Negative TPRQ—8th interview до > .20 
Positive TPROQ—4th interview -73 >.20 
Positive TPRQ—8th interview .21 2.20 
Change Variables; 
Maladjustment Index 1.22 >.20 
Dependency .79 >.20 
efensiveness .16 2.20 
Positive Attitudes Toward Self .87 2.20 
Positive Attitudes Toward Others .48 >.20 
aylor Anxiety Scale ‚бо 2.20 
Therapist Posttherapy Rating Scale «42 >.20 А 
* All analyses were made on the difference between pre- and post-therapy scores except for the 
Taylor and Posttherapy Rating Scale where postthei 


in each of the therapies affected the de. 
gree of defensive reactions of clients, 
Such effects are pronounced by the fourth 
interview and persist at least through 
the eighth interview, Client positive reac. 
tions tend to vary in the same way. By 
the eighth interview, differences i 
positive reactions are clear 
therapist reactions do no 
interaction effects, None 
verbal behavior vari 
tended to reflect ch 
revealed any signi 
fects. 

"Therapists interacting with the ther. 
apy they are administering Apparently 
create different effects in the Positive anq 
defensive aspects of the relationship from 
clients’ points of view, The defensive 
aspects of the relationship tend to be. 
come less related to therapists aS person, 
but remained related to the interaction 


n client 
“cut. However, 
t show similar 


of the client 
ables or variables in. 


ange during therapy 
ficant interaction ef- 


тару scores were used, 


of therapist with the type of therapy vi 
ministered, On the other hand, the post 
tive aspects of the relationship beron 
more clearly related to therapists гй = 
dividuals as Well as to therapists inte? 


А B га ad- 

acting with the type of therapy 

Ministered as therapy progresses. ;an 
Inspectio a 


n of the raw data suggest dë 
se changes. On the di 
total means for p 
therapist tend to converge toward nie 
total mean for all therapists wx 
the fourth and eighth interviews. At t 


: the 
Same time, the difference between ans 
reflective an nes 


for each 


explanation of the 
fensive CPRQ, 


d leading treatment ! ome 
therapist tends to bee pth 
Sreater between the fourth and €8 
interviews, Thi 
Progresses clie 
Selectively to 
hanges o 
Primarily f 


S suggests that as ue 
nts tend to react "n 
their therapists’ ge^. 
n the positive СРЕО  ! 


- гог t€ 
Tom changes in the erro 
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in the F ratios. At the fourth interview, 
the error term accounts for about one- 
third of the total variability, while at 
the eighth interview it accounts for only 
about one-eighth of the total variability. 
In other words, the scores of pairs of 
clients for each therapist in each treat- 
ment condition became more alike be- 
tween the fourth and eighth interviews, 
while differences among averages of the 
scores for each therapist and each ther- 
apy tended to remain the same. This sug- 
gests that clients’ subjective reactions 
tend to become less influenced by their 
habitual interpersonal sets and more 
by factors within the therapeutic situa- 
tion. 

A generalized fear of intimate inter- 
personal situations may be one of the 
more important of these sets. As this gen- 
eralized fear is diminished, the client be- 
comes more discriminating in his reac- 
tions. Because the client is reacting more 
discriminatingly to the stimuli within 
the situation, a more favorable set of 
circumstances for therapeutic change 
would appear to be developing. 

Several factors may be related to these inter- 
action effects. Therapists’ training and experi- 
ence may influence the effectiveness with which 
they administer different therapies. In this re- 
Bard, it is interesting to note that the two 
therapists with the most extensive training and 
experience with leading forms of psychotherapy 
obtained the highest defensive CPRQ scores in 
the leading treatment at both the fourth and 
eighth interviews. Therapists’ own dynamics and 
personal value systems may relate to the way in 
Which they make use of a type of therapy. The 
types of roles assumed by therapists may vary 
with the type of therapy offered. This study 
provides no strong clues toward determining 


which factors are significantly related to these 
differences. 


General Discussion 

This study indicates that clients’ views 
of the therapeutic relationship depend 
Оп the interaction of the clients’ own dy- 
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namics, the kind of therapy ad- 
ministered, and individual characteris- 
tics of therapists. This idea has been 
discussed frequently in the literature. On 
the other hand, the idea that one type 
of therapy is good for all clients and that 
any therapist can be effective with a 
given type of therapy has also been sug- 
gested. The present data tend to support 
the former rather than the latter view. 

The implications of these results seem 
of considerable importance. Extensive re- 
search is needed to define differential 
client reactions to different therapies and 
to different therapist characteristics. Re- 
search is also needed to define the client 
pretherapy characteristics related to dif- 
ferential client reactions, as well as re- 
search relating client reactions during 
therapy to therapeutic outcomes. 

The consistently significant findings 
on client guarded or defensive verbal be- 
havior and the client positive and de- 
fensive relationships measures also sug- 
gest that these measures are sensitive 
to some of the effects of therapy and 
therefore merit further study. This ob- 
servation may have broader implications, 
however, for research in this area. If one 
wants to measure what happens in ther- 
apy, behavior in therapy or reactions to 
the therapy situation specifically may be 
one of the most sensitive areas of meas- 
urement. 

This study clearly illustrates the value 
of multivariate experimental designs 
with measures taken at different times 
when studying as complex an area as 
psychotherapy. The writers firmly be- 
lieve that major advances in the effective 
use of psychotherapy will come most 
quickly from carefully designed and ex- 
ecuted research. Only in this way will 
therapists become able to administer 
therapy based on a body of verified 
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knowledge rather than finding it neces. 
sary to rely primarily on individual clini- 
sa 

cal experience. 


CONCLUSIONS 


The view that a leading and a reflec- 
tive type of therapy produce different ef- 
fects ‘on clients was slightly supported. 
During the first eight interviews, these 
therapies did not produce significantly 
different effects on the defensive or posi- 
tive aspects of the relationship from the 
clients’ points of view nor in the posi- 
live or negative aspects of the relation- 
ships from the therapists’ points of view. 
The therapies did not differ significantly 
in the extent of defensive, dependent, 
open, covertly resistive, or overtly 


resis- 
tive verbal beh 


avior elicited from clients 
during the first four interviews. The 
therapies did not differ 
the amount of client 

therapy in dependence, 
maladjustment, positive att 
self, positive attitudes toward others, or 
anxiety. Of 21 variables explored, client 
guarded verbal behavior and the Thera- 
pists’ Posttherapy Rating Scale were the 


only ones showing a Statistically signi- 
cant difference, 


The view that 


significantly in 
change during 
defensiveness, 
itudes toward 


pretherapy characteris. 
tics of clients relate differentially to 
client reactions to therapy in reflective 
and leading types of therapy is partially 
supported. There were differences | 
tween the two types of 
manner in which 
ness, and prethera 
lated 


be- 
therapy in the 
pretherapy defensive. 
PY aggressive need re- 
to verbal defensive b 
clients in therapy, There w 
ferences between the tw 
apy in the manner in w 
needs to be deferent an 
mous related to client s 
sive reactions to ther 
differences in the oth 


chavior of 
еге also dif. 
9 types of ther. 
hich pretherapy 
d to be autono- 
übjective defen- 
ару. None of the 
€T 20 sets of corre- 
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lations were statistically significant. The 
differences between treatments reflect the 
following findings. Clients who were 
more defensive when they entered ther- 
apy tended to behave more defensis e 
when they were in leading therapy. 
There was no such relationship under ut 
flective therapy. There were tendencies 
for clients who entered therapy with 
more aggressive need to behave more > 
fensively in the leading treatment, anc 
less defensively in the reflective therapy. 
Clients who entered therapy 
need to be deferent to oth felt more 
defensive in the leading tr ment. мај 
such relationship Was apparent in oe 
reflective treatment. Clients who entered 
therapy with a strong need for autonomy 
tended to feel less defensive in leading 
therapy. 


with more 


Autonomy need seemed us 
lated to defensive feelings in the reflec- 
live treatment, cg 

The view that. individual vais ind 
create different effects. on their po 
independent of the type of therapy ү! 
is partially supported. ‘Therapists ihe 
this study differed significantly in v 
defensive and positive feelings they € ak 
ited from their clients during the ne 
eight interviews, They also differed s18 
nificantly in the 


8 rer- 
extent of guarded Y 
bal behavior 


exhibited by their B 
in the first four interviews. pe 
they did not differ p^ е 
extent of € ' 
covertly 


significantly 
defensive, dependent, istive 

resistive, or Hes in 

verba] behavior 

the first four interviews. T 

differ Significantly in their view о rel 

positive ог negative aspects of ihe јелу: 

tionship during the first eight interv 

They did not 


„a th 
ж 1 
differ significantly "eit 
extent of change produced in 


ces 
1; Р den 
clients in maladjustment, Череп ard 
defensivene 


tow’ 
self, positiv, 


overtly 


és 3 -[ients 
elicited from cl т not 


35, positive gaitue а d 
Р: 1 
е attitudes toward ot 
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anxiety during the course of therapy. 
They did not differ significantly in their 
evaluation of the extent of change pro- 
duced in their clients as a result of ther- 
apy. 

The view that selected therapist char- 
acteristics ате related to the kinds of re- 
lationships established, the amount of 
defensive or guarded verbal behavior 
elicited from clients, and the amount of 
change in adjustment produced in 
clients is not supported. Therapists’ abil- 
ity to enter the phenomenological field 
of another, sympathetic interest, ac- 
ceptance of the value system of others, 
need 
eness 


social stimulus-value to associates 
to aggrandize the self, and aggres 
did not correlate significantly with any 
of the five dependent variables ex- 
amined. 

"The view that the interaction of the 
therapist as an individual and the type 
of therapy he is employing affects clients 
is partially supported. The way in which 
therapists used or molded a type of ther- 
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apy had effects upon clients. Clients felt 
significantly more defensive or more posi- 
tive in one type of therapy with indi- 
vidual therapists than did other clients 
for the same therapist in the second type 
of therapy. For some therapists the in- 
defensive or positive feelings 
were in the leading treatment while for 
other therapists the reactions elicited 
were greater in the reflective treatment. 
The clients in this study did not differ 
significantly as a result of the interac- 
поп effect (therapist X method) in the 
extent of guarded, defensive, dependent, 


creased 


open, covertly resistive, or overtly re- 
sistive verbal behavior which they mani- 
fested. They also did not differ signifi- 
cantly as a result of the interaction effect 
in the extent of change they exhibited 
in maladjustment, dependence, defen- 
siveness, positive attitudes toward self, 
positive attitudes toward others, anxiety, 
or therapists! evaluation of change as a 
result. of. therapy. 
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APPENDIX 


THE CLIENT'S PERSONAL 
DIRECTIONS 


During the process of personal counseling, 
people have many different feelings and reac- 
tions. We know that these reactions are some- 
times negative, sometimes positive, and often 
mixed. Your responses to the following ques- 
tionnaire will help us understand people's re- 
actions to personal counseling, "This will have 
nothing to do with your counseling, It will be 
completely confidential, Neither your counselor 
nor his superiors will be informed of your re- 


sponses. Approximately 15 minutes are required 
to complete it. 


There are five possible res 
the items in the questionnair, 
1 not characteristic 
slightly characteristic 
moderately characteristic 
quite characteristic 


5 highly characteristic 
Put a circle around the r 


sentative of your present fe 


Ponses to each of 
е, 


4» CO ы 


л 


еѕропѕеѕ most терге- 
elings. Your feelings 


REACTION QUESTIONNAIRE 


may have been different in the past and may be 
different in the future, We are interested e 
your feelings right now at this point in Pie 
counseling experience, Be sure to put a argie 
around one response for each item. Do "® 
spend too much time on any one item. 


POSITIVE ITEMS! 

1. I'm pleased with 
tion. 
wish I felt as 
as I do here. à 
is 1 "T any” 
3: T like our sessions even when Т can't think of 

thing to зау. 
4. I remember and “ 

selor says, 


i eae а Very warm feeling toward my coun 
wish I had some friends who were as unders 
as my counselor. 


7. Тат usually ед, 
Say. 


^e айе бее] fic Бернес cor anier ur intervie’ at 
Sometimes feel like letting my counselor know 


à nice person I think he (she) is. 
ieee 


В atten- 
! my counselor's interest and 


м 


ions 
+) situations 
Sure of myself in all social situ 


E coun- 
chew over" things that my 


selor. 
tanding 


to 
has 
ger to hear what my counselor 


1 i 
Grouped for the reader's convenience. 
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to. I wish I could feel other people respected and liked 
me as much as my counselor does. 

1. Т feel comfortable talking with my counselor. 

2. This is one of the few situations I've ever been in 
that I didn't worry very much about what the other 
person thought of me. 

13. D wish I were more like my counselor. 

14. I feel that my counselor regards me as a likable per- 

son. 

15. Many of the things my counselor s. 

hit the nail on the head. 

16. 1 experience a certain re 

selor something. 

17. My counselor's attitude gives me hope I can get some- 

thing out of this. 

18. I feel that the counselor really likes to spend the 

counseling session with me. 

10. I feel my counselor is really anxious to help me solve 

my problems. 

20. It's easier for me to talk with this counselor than 

with most other people. 

21. I think I could criticize or get angry at my counselor 

and he (she) wouldn't resent it, 

My counselor must be one of the best ones here, 

My counselor's understanding of me is encouraging. 

usually feel the interviews have been worth while. 

I really get “wrapped up" in what's going on in the 

counseling session. 

26. I wish I had asked for this kind of help soone 

27. I can talk about most anything in my interviews 

without feeling embarr: lor ashamed. 

wish I could spend more time with the counselor. 

am gaining more respect for psychology as a result 

of my experiences in counseling. 

зо. The things my counselor says and does giv 

fidence in him. 

31. I seldom feel the counselor has misinterpreted what 

I have said or done. 

32. I know the counselor understands me ev 

don't express myself well, 

33. I have the feeling here is one person Т 

trust, 

would like to behave toward other people more like 

my therapist behaves toward те. 

35. I look forward to talking with my counselor. 

36. I feel sure that my counselor would take anything 
, 1 could say or do without getting upset. 

37. I'm pleased with the progress I've made since begin- 

ning these interviews, 


I'm glad this particular Counselor was assigned to 
me. 


ys just seem to 


ief after telling my coun- 


m 


е me con- 


en when I 


c 


1 really 


w 


39. The counselor i 
40. I think my cour 
difficulties, 


à warm and friendly person. 
nselor really sympathizes with my 


DEFENSIVE ITEMS 

1. Tt would be helpful if I could write things down and 
bring them in to the counselor to discuss. 

2. I doubt if many people get much help out of these in- 

terviews. 

3. Sometimes the counselor seems to twist around the 


things T say to mean something different than what 
I intended. 

4. T's hard for me to talk about myself. 

5. I sometimes feel like calling this whole thing off. 

6. I've told the counselor a lot but he (she) still hasn't 
Riven me much help. 


THE THERAPIST'S PERSONAL 


DIRECTIONS 


During the process of personal counseling, 
counselors have many different feelings and re- 
actions, These reactions are sometimes negative, 


DING TYPE OF PSYCHOTHERAPY 25 


7. It takes a long time in an interview for me to get 
started talking about important thing: 

8. It seems to me there should be an easier and quicker 
way than this to solve my problems. 

о. Many of the things we talk about don’t seem to be 
related to my problems. 

10. І sometimes feel like leaving before the interview 
hour is over. 

11. Sometimes after the counselor says something I just 
can't think of anything else to say for awhile. 

12. I sometimes hesitate to tell my counselor what I am 
really thinking. 

13. If I had someone else as a counselor, I would prob- 
ably feel freer to discuss my problems. 

se interviews seem like a waste of time to me. 
15. Sometimes I feel like I'm being criticized during the 


10. It doesn't make much difference to me whether my 
counselor likes me or not, 
17. I frequently find it difficult to think of things to 


18, I get irritated at some of my counselor's comments, 

19. I spend very little time think: ng about these inter- 
views when I'm not here. 

го. It was kind of unnecessary 
seling because my prob! 
major. 

s1. I just don't know what to do or say in the inter- 
views that would help. 

22. When I'm the counseling session I sometimes 
forget things I had meant to tell my counselor. 

73. If my counselor understood me better I could make 
more progress. 

24. I try to justify my actions so the counselor will see 
why I behaved the way I did. 

There are some things which I don't yet feel ready 

to go into with my counselor. 

26. T feel a need to keep the conversation moving during 
the counseling hour. 

27. l can't see where my counselor has done much to help 
me solve my problems. 

don't know exactly why, but I feel nervous about 

coming to the counseling hour. 

20. Sometimes it's hard for me to pay attention to what 
the counselor is saying. 

30. I know what my problems are, but I don't know what 
to do about them. 

31. I carefully organize what I'm going to say in the next 
counseling session. 


The counselor's looking at me all the time makes me 
uncomfortable. 


33. I sometimes wish we were talking 
different than what we're talking 
ment. 

34. If I would take a couple of hours а м 
about these things on my own, I could probably ac- 
complish as much as I do in these interviews. 

35. I feel like my counselor wants me to tell him a lot 
more than I am telling him. 

36. Tni afraid to express my real feelings in these ses. 
sions, 

37. I don't think T have as m 
ple in counseling. 

38. I 


Sometimes resent the counselor attitude toward 
h lor's 


39. I sometimes feel like I'm bei 
39. I feel that my counselor h; 
Sorized as some kind of ** 


for me to start this coun- 
lems really aren't very 


about something 
about at the mo- 


eek to think 


any problems as other рео- 


REACTION QUESTIONNAIRE 


ару seem to agree 
and reactions toward 
as long as the coun- 
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selor recognizes and understands them. In fact, 
a rovide additional sensitive. evidence i ; ; 
ved T4 pensa of a client's communications. would feel in the situations he describes than it 
about the m a д ee ВБ 
i i v esc fecl- is with others, — 4 : И 
WE ате interested im нше ub. 18. І sometimes feel like congratulating this client for 
i ку ж ing he has done. 
are and how they chang А | RSE RE bee MORE | | Н 
а responses will have nothing to do with тиа more aborted in what this dieit is do: 
i "ai х than wi ers I've had. 
i ' will not be made avail- ing than with others I've . 
ur жол н. Др visor 20. Responses to what this client is saying come more 
able in identifiable form to your supervisor or го. Respon: A dos ES 
anyone else except the researchers. h of have our ups and downs, but underneath 
* ^ i nses to each ol 
There are five possible respor 


it all I think the client has confidence in me. 
the items in the questionnaire, Therapy with this client is a more rewarding ex- 
1 not characteristic 


perience for me than with many others I've had. 


16. We get into more important material than is fre- | 
quently the case with other clients. д г | 
17. It's easier for me to see exactly how this client 


— 23. When things are not going well for the client I feel ] 
2 slightly characteristic. А canted oar д | hs 
3 moderately characteristic 24. I usually have a good feeling about interviews with 
i haracteristic this client, — | . А 
сд is ЕН 25. T think I'm doing a pretty competent job with this | 
5 highly characteristic client: 


Put a circle around the responses most repre- 
sentative of your present feelings. Your feelings 


^ 


think we have a pretty relaxed, understanding kind 


may have been different in the past and may be 
different in the future, We are interested in your 
feelings right now at this point in your counsel- 
ing experience with this client. Be sure to put 
a circle around one response for each item. Do 
not spend too much time on any one item. 
The numbers may appear in different orders 
before cach item, but they will always signify 
the same response. Some of the items are writ- 
ten comparing this client with other cl 
have had. Try to respond to the items 
of that comparison, if possible. 

When you have completed the questionn: 
return it to the envelope and drop the 
envelope in the box marked TESTS i 
ing room. The test must be com 
your next interview with this clie: 


ients you 
in terms 


aire, 
aled 
n the wait- 


pleted before 
nt. 


POSITIVE ITEMS? 
1, I like this client more than most. 
2. I have a more warm, friendly emotional reaction 
toward this client than others, 
3. I feel sure that this client would not w. 
therapists if given the chance, 
4. D am seidom in doubt about wha 
to say. 
5. I think about this client more 
ings than others. 
6. This client seems to appreciate my efforts, 
7. In general, I could not ask for a better client, 
8. ГИ miss having these interviews a little when the 
client decides to terminate. 
n. I prefer working w 
I've worked with. 
то. If T rated all of the clients Тус wor! 
in my career in terms of the s 
ten out of them, this client wi 
rating. 
11. T think this client is trying harder to solve his (her) 
problems than others I've had. 
12. T sometimes wish the ther 
would not end so soon, 
13. I am more confident this clien 
problems than I've been with 
14. T can usually find significant thi 
what the client says, 
15, I think I'd like this clie 
first in that capacity, 


ant to change 
t the client is trying 


often between meet- 


ith this client more than others 


Кей with so far 
atisfaction I've got- 
ould receive a high 


apy sessions with this client 


t will work out his 
others, 


ngs to respond to in 


nt socially if T had met him 


* Grouped for the reader's con mente, 


- I look forward to my 


» Sometimes I 


. I don't feel the ¢ 


. l can’t get close to thi 


2. It is really 
3. This client * 
. I find it harde 


of relationship, 


I feel more comfortable in the therapy sessions with 
this client than with others I've 


- I find it easier to understand and communicate with 


this client than with others. vods 
Аз compared to others, I'm pretty “wrapped ир" in 
this client and in try ng to help him. Р 
If I had to leave here for some reason before this 
client was finished, I'd try very hard to see that 
he was assigned to “Just the right therapist. 
I'm glad this particular client was assigned to me. 
1 get anxious about what to do or say with this 
client less frequently than with others I've hwl 
interviews with this client 
more than with others I've had. А 
Similarities between my own emotional experiences 
and some of this client's make me feel a little 
closer to him (her) than to others I've had. 


elationships like this are bright spots in my 
schedule, 


NrGATIV 
I'm usually relieve 
1 can't get this clier 
І get pretty 


ITEMS 

d when the intervi 
At to open up. 

bored in some of these interviews. — 
Ket pretty tense during the interviews- 


I seldom feel t we have accomplished something in 
the interviews, 
I 


would like to be able to fe 
this client than I now feel. 


ws are over. 


el more warmth toward 


lient is making as much use 0 
therapy as he could, 


I don't Particularly 


В я is client. 
enjoy my hour with this clien 
5 client. 

4 ; i t 
if another therapist might no! 
s client than T. 


client about. some ba 
» morality, etc. 


an effort to “ ay with” this client. 

"hits me where it hurts" sometimes. d 
Т to remember what has been cover 

interviews with this client than wit 


sometimes wonder 
get further with thi 
I disagree with this 


jc matters 
like religion, 


in previous 
others, 


| E „лей ê 
In comparison with other clients, T find it hard 


get involved with this client's. problems. 
I feel in need of help with this case. 


can’t seem to get very interested in this client. 


1 ^ his 

Seem to tire more quickly when I work with t! 
client than with others, 

I sometimes f 


Á feel “pushed” by this client. 
- Sometimes [ 


: inter- 
5 feel Pretty frustrated in our if 
views, int 
à strait 
l have 10 Exert more self-control and self-rest™ 
with this client than with most. 


1 doubt if any counselor could do much for 


this 
client. 


EFFECTS OF 


з. I get pretty discouraged at times about this one. 

4. I feel pretty ineffective with this client. 

5. l prefer working with this client less than others 
I've worked with. 

26. I can't help but be annoyed to some extent by some 

of this client's behavior. 

I'm "flying blind" with this client. 

‚ Sometimes 1 resent the client's. attitude. 

20. It's hard to know how to respond to this client in a 
helpful way 

30. Та comparison. with other clients, it's hard for me 


CLIENT VERBAL BEHAVIOR CATEGORIES 


GENERAL. RULES 
A client ге 


ponse is a client statement or 15 seconds 
of silence between two therapist statements, If a client's 
statement is longer than 15 seconds it is divided in 15 
second units, The last part of a client statement between 
a previous 15-second portion and the time the therapist 
ks is categorized even though it may be less than 
15 seconds long, unless it is a pause, 

The category “N" is used for the client only when 
there is nothing in the client's response to call for 
classification anywhere else. 

3. If a statement cannot definitely be placed in a 
category other than “N,” because it is not complete at 
15 seconds when the recording machine is stopped, put 
category, But its meaning is allowed to 
e bearing on the categorization of the next 13-second 
portion of the client's response. 1f, on the other hand, 
enough of the client's statement is heard to put it defi- 
nitely in а egory, do so even though the sentence ma 
not be complete, 

4. The same sentence cannot be categorized in two 
s (unless it is an. Interruption, Blocking, or Change 
ic), but more than one category may be used for 
-second interval, 

5. If in conflict between two or more categorizations 
of the same client sentence, place it where the | i 
ference is required. 

6. We are interested in the variables being measured. 
here only as they are manifest in the counseling situation 
itself, Client statements. which indicate only that the 
client is or has been, for ex: mple, guarded or dependent 
outside of the therapy hour do not fall under the € 
edness or Dependeney categories here. 


st in 


DEPENDEN 


iy 


Briefly defined, “Dependency” is the extent to which 
the client asks the therapist for his opinions, or advice, 
information, evaluation, instruction, or demonstrates a 
need for structuring from the therapist. Also it includes 
the degree to which the client places responsi bility for 
Progress or outcome of counseling on the counselor 
rather than accepting it himself. 

Statements to be categorized as Dependency аге state- 

ments in which the client: 
«s for help, opinions, advice, solut ns, informa- 
tion, judgments, evaluations, instructions from the thera- 
pist, Asking the therapist for elaborations on his state- 
ment so included here. 

2. Shows by his statement that he is putting the re- 
sponsibility for providing the above-mentioned things on 
the therapist, or he puts the responsibility for progress 
or outcome of therapy on the therapist. This can be 
shown indirectly through such statements as "You won't 
һе able to figure out what's wrong with me unless I 
keep talking." 

Note: Statements pertaining to appointment time, and 
asking for cigarettes, match, ete, are not categorized 
here. А 

Rhetorical questions (e.g., “Did I tell you аһош...”) 
are not categorized here, Some pause or other indication 
that the client expects an answer is necessary for client 
questions of this sort to be categorized as “Depend- 
ency," 
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to put myself in this client's place. 
31. l am sometimes at a loss as to how to respond to 
this client. 
I don't think this client will stand out in my pl 
memories of cases. 
33. Sometimes | have to show more sympathy and ас 
ceptance than D really feel toward this client. 


ant 


34. Sometimes 1 wish some other therapist had this 
g nt. 

35. The hour often seems to be dragging on with this 
client. 

On the other hand, there may be statements which 


are clearly questions. being asked of the thera 
though they may not be put in question. form 
ample, "I wonder why that would be (pause), 
a statement seems aimed 
categorized as "Dependence 

The following statements 
the category of Dependen 

1. "D didn't expect this to be ea 


al 


If such 
at the therapist, it should be 


are some examples to clarify 


y or direct, but it 
1 be accomplished here 


apist or the 


ару a Covert 


се ion would be warranted in addi- 
tion to a Dependency categorization. 
Therapi “She didn't want to.” Client: “You 


my sister?" , Х. А question. whieh 
ation of an ambiguou. 
а Dependence 


calls for 
therapist statement does not 
wization, lf the therapist's 
"ment judged clear in meaning and the client 
nevertheless requests. clarification this may receive а 
Covert: Resistance categorization, 

з. Therapi “Maybe you could do some thinking 
about that." Client: "In what way?" S D. The client 
is asking for instruction, 

4. “When do you think 
ing about getting married? 
for therapist's opinion, 


person should start think- 
.. D. This is a request 


OPENNESS 

“Openness” is defined as: the extent to which the 
client freely discusses his problems, deviations from the 
"his culturally frowned-upon traits, behavior, 
and motivations; and, in general, his willingness to ex. 
pose himself to potential criticism and change, partieu- 
larly his willingness to discuss thoroughly those areas 
which seem most threatening. He does this without at 
the same time qualifying, hedging, and engaging in de 
fensive verbal maneuvers, 
In a way th ategory is the opposite of the “guarded. 
ness" category, When the client's statement does not ex- 
hibit any guardedness in a discussion area in which a 
clinician would often expect a person to be guarded, the 
Client's statement is categorized under “Openness,” This 
пецогу is meant to reflect the ability or Willingness of 
client to “open himself ир” to the counselor; to e 
himself to possible ticism, 
prospect of modifying his self concept, without at the 
time feeling the need to defend himself ах he is to hi 
or to the counselor's eyes, А confidence in the thera 


understanding, noncritiealne: 5 and trustworthiness is pre 
mness; i.e. the statements that 


sumed to underlie sucli о 
are considered "open" are not such їз one would tell to 
rides or even new-found friends, 


one's 
"qualified? admission to a problem,* 


1 
spose 
to expose himself to the 


T 


deficiency, in- 


3An admission to 


; а problem is “quali "n 
is coupled with ied 


2 он п expression of uncert 
be: perhaps, "probably," etc; the term “I think 
howe not considered to indi 
ery indicate uncertainty 
Ж Ted ainty), 
Suelinesittor may also exist in any minimization of А 
o»'em in terms of the extent E i 
erms е› Or severity of à 
m Such minimization is of course, a different thi 
à us 2 P ce Me 
an сапа ion and specification, and the coder indt 
ane’ А decision as ta whether the енене is pe 


when it 
ainty (e.g, “may- 
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adequacy, undesirable characteristic, trait, behavior pat- 

tern or act, fcelings or attitudes. К { ы: 
2. Unqualified statements pointing to porsona 

viation from the norm in a culturally or personally un- 

А Башы: | 

op re statements that admit to possession of 
culturally or personally undesirable characteristics, 
traits, behavior, feelings, or attitudes. ae е 

Note: Simple acceptance of a therapist's emnt 
placing the client in an undesirable light is not score 
“Open.” It is only when the client proceeds in such a, 
way as to place his statement under any of the. criteria 
listed above that his statement is categorized. Open. 

A simple statement of a problem qualifies as an 
“Open” response if it is unqualified, h А 

A description of a particular conflict is not necessarily 
“Open.” But an unqualified admission of having | im- 
portant unresolved contradictions or irrationalities within 
oneself is categorized “Open.” 

The following aene a some examples to 

ify the categorization of “Openness”: 
bua pai about husband or father): “As 
far as really deep feelings—I have none for him.” > 
О. This is an unqualified admission to culturally unac- 
ceptable attitude or feeling. 

2. “My conversational ability is pretty weak, I can’t 
carry on a long continuous conversation." . . , О, This 
is an unqualified admission of an inadequacy. 

3. Therapist: “That seems kind of contradictory.” 
Client: “When I think about it, that’s true; as far as 
things have gone in the past I have no reason to feel 
inferior.” , . , O. The client accepts and elaborates 
upon therapist statement pointing to important contra- 
diction ог irrationality in client. 


4. "Most of the time I'm worried about people—what 
+ + O. This is an unqualified 
Problem, 


GUARDEDNESS 
Guardedness is defined as the extent to which the 
client exhibits wariness and hedging in regard 10 
presenting and working on his problem, admitting to 
faults, and exposing himself to potential criticism and 
change. This includes self-stimulated denial, or mini- 
mization, of his Problems or his deviati 
"normal"; and denial of cul 
traits, and motivations, It 
justify himself or his 
pectations or anticipati 
apist. 


| Өн judge to ask hi 
ing to a client’s statement with ref 


gory is whether the client is in 


mself in listen. 
erence to this cate. 


, s any way engaged in 
“protecting” himself from potent; A 


1 ial criticism or po- 
tential change in his self Concept. Some of the cues to 
listen for are presented below, 

1. Statements denying, qualifying, minimizing, or 
belittling the extent of a Problem or t 


he existence of 


tious and guarded, or merely giving some definitive in- 
formation to his counselor, When the case is ambiguous 
to the coder he should not categorize it as “Openness” 
(or Guardedness) but as “None.” A qualification must 
occur in the same 15-second interval as a statement in 
order for it to disqualify the statem. 


an admission is accompanied (i 


response. 


4 The word “problem” js defined ir 


"Guardedness," 1 the section on 


КМЕҮ 


one The denial is not in response to a question or 
statement of the therapist or any other particular per- 
son. T К 

2. Statements pointing to nondeviation from the 
"norm," "average," “everyone,” "other," etc., either 
as a person in general ог in some particular aspect of 
thought, feeling, or behavior (c.g., “I guess we all have 
a tendency to talk іп сїгсїез.”). If such a statement is 
purely descriptive and factual or is in the nature of a 
complaint, rather than being sometl which is com- 
forting to the client, it does not receive a Guardedness 
categorization, 

3. Statements denying possession of an unde 
characteristic, trait, feeling, attitude, or den 
undesirable act or motivation, This category is not used 
when the characteristic has been attributed to the client 
by the therapist or some specific other person. кыз 

3. Statements in which the client attempts to justify 
an act, thought, feeling statement, etc, to the therapist. 
“Justify” here means to hold forth one's behavior as 
Just, right, warranted; to declare oneself guiltless; ab- 
solve or acquit oneself; to attempt to show isfactory 
excuse or reason for something that is culturally or 
Personally undesirable, The reason that is offere UY 
the client is usually one that is primarily “outsi > of 
the self, ie, the undesirable thing i result of the 
behavior of others or circumstances, "This category does 
not apply if the client has been asked to justify himself 
by the therapist or some specific other persons. А 

5. А statement indicating that the client might be 
anticipating a critical or differing thought or statement 
from the therapist, (For example, “That’s the wa Л 
think about it, you тау think differently, I don't know"). 
Such phrases as "sound to you" or “look to you” iam 
the nt to the therapist should alert the coder to T 
Possibility that such a statement is an anticipation of dif- 
ference or criticalness between the client and the coum 
selor. 1 

Some such statements could serve to beat the 
therapist to the punch. “All this must sound foolish v 
you," Here the expectation by the client of a critical 
attitude on the therapist's part is overt. In some C 1 
the anticipation of a critical attitude is only implied 
by the client's overt acceptance of the blame or fault- 
finding he expects from the therapist, He will thus 
make a statement taking fault or responsibility upon 
himself, while he is in reality rejecting the blame La 
criticism, Sometimes this takes the form of a mu^ 
qualified acceptance of a Statement by the therapist d 
Some other person which places the client in а ше 
light. Admitting to blame does not, of course, шд 
matically indicate that the statement should be cate 


; Jong 
Rorized as Guardedness, Such a statement may belong 


in the N or ОР Category depending on the way it 1% 
said and: 


1. The extent of qualification, As a rule, in doubtful 
cases, do not categorize as G an unqualified arate 
accepting blame (eg, "I'm being foolish.”) Do ali- 
categorize as G a statement containing a single au% 

cation (eg, “Maybe I'm being foolish.") Do еа 
ЖОНЛЕ ай G sub à віне ніс авга ш 
number of qualifications (eg, "I guess maybe 
being foolish") Él 

2. The nature of 
that is found withi 


ment 
the elaboration on the stater 


s à 
s n the same client response. he 
rule, in doubtful Cases: (a) the latter part St ihe 
client’s response is considered more important—! 


— ich 
, ° “Problem” is broadly defined here as anything whic 
is culturally frowned.upon, or which bothers th 
personally, То qualify under the latter the client ш 
make his personal distaste clear, Physical symP' 
or manifestation: 


Sweating, shaki: 
are not consi, 
"problem," 
ness, feelin, 
“Problems,” 


vous" 
But psychological symptoms (e.g. d d Ld 
g depressed, etc.) are considere 


— 0) 


EFFECTS ОЕ А REFLECTIVE AND 


follows the blaming of outside 
it should probably not 
ategorized as G. On the other hand, if the blaming 
of outside factors follows self-blame, this should prob- 
ably be categorized as G; (b) the more detailed and 
specific part of the client's response is considered more 
important and the vague, more general, part considered 
to be less important. Thus, if a client states he is at 
fault in some general w but c izes another person 


or a circumstance in a more specific way, the state- 
ment is likely to be a Guardedness statement. 
Note: A simple rejection of a therapist's statement, 


whatever it may have been, is not 
It is only when the client proceeds in such a way as to 
his statement under any of the criteria listed 
above that his statement is categorized as Guardedne 

The following statements are some examples to cla 
the categorization of Guardedness: 

1. “I wonder if I have any problems. Maybe it's 
just that I think I have problems, Maybe that’s all 
there is to the whole thing." . . . G. This is minimiza- 
tion of the problem. 

2. “I guess everybody feels 
thing.” . 0. . G. This is 
nondeviation from the norm, 

3. "I was proud of the medal, and I showed it to 
everyone, as every successful athlete would." , , . G. 
The client is pointing to nondeviation from the norm 
in regard to the culturally frowned-upon trait of pride. 

4. “I don't like to visit my family because when I 
ve too much work to do it bothers те”... С. This 
is justification, 

5. Therapist: "You feel inferior to them." Client: 
"Maybe that's true. I don't know. Anyway, they kept 
talking about things with which I wasn't fami r, which 
I thought was very inconsiderate of them." . . . If the 
manner in which the statement is said is consistent 
with such a categorization, this doubly qualified ac- 
ceptance is put in the G category as an overt acceptance 
of a therapist's statement placing the client in an un- 
desirable light which is really emotionally rejected by 
the client. . 

6. Therapist: “You feel inferior to them.” Client: 
"No, it wasn't that, It was just that they kept talking 
about things with which I wasn't fami r, which I 
thought was very inconsiderate of them." . . ‚М.А 
rejection of a therapist’s statement placing the client 
in an undesirable light is not categorized G unless it 
also in one of the criteria listed for G. 


categorized as G. 


y 


that way about some- 
self-stimulated pointing to 


- COVERT RESISTANCE 


Covert resistance is defined as the extent to which 
the client manifests indirect or impersonalized criticism 
of the therapist or therapy, also blocking, delaying 
tactics, failure to recall or report things, changing the 
subject, interrupting the therapist. It is resistance or 
hostility toward therapy, therapist, progress in therapy, 
or toward things which are thought of as being con- 
ducive to such progress. But the resistance is not di- 
rectly expressed verbally; instead, other more subtle 
escapes or hostilities are resorted to by the client. 

т. Long Pauses (LP): А rs-second "response" in 
which the client does not talk. 

2. Short answer (SA): Ап unelaborated simple 
thought statement not longer than four or five words 
that is followed by a pause. This category is not used 
when the client's response is in reply to a question by 
the therapist which can be given a simple affirmative, 
negative, or factual answer. 

3. Changing Topic (TC): Client initiated changes in 
the topic being discussed. The new topic is clearly un- 
related to the previous statements of the counselor or 
client. Client statements which serve to cut off a topic 
(e.g., "That's all I have to say about it.") are also in- 
cluded here, 

4. Blocking (ВІ) 
the product of an 
tracing and rephrasi 


Incomplete Sentences that are not 
interruption by the therapist, Re. 
ng or fumbling of sentences, Pauses 


A LEADING 


TYPE OF PSYCHOTHERAPY 20 


in mid-sentence (at least 5 seconds in length). Any 
statement of, or indication of, failure or inability to 
think or talk about something in particular or anything 
in general nability to recall a word, situation, or 
example; inability or failure to give reasons or motiva- 
tions; inability to give any label to one’s own feelings, 
attitudes, or perceptions. 

5. Interruption (Int.): Client interruptions or over- 
riding of therapist's verbalizations. (Agreement in- 
serted into a therapist's verbalizations with no intent 
to cut him off are not included here.) 

6. Verbalization or intellectualization (V): Talking 
about minutiae or irrelevant details. Abstract discus- 
sions of politics, religion, etc. Such statements bear 
no discernible relationship to the client's problems as 
he has been expressing them in the interview, To qualify 
for a categorization here such a statement must take 
up a whole 15-second interval. However, if the client 
spends time groping for an irrelevant detail his re- 
sponse is categorized here whether or not the whole 
15 seconds is spent on irrelevancies. 

7. Resistance toward therapy or therapist (Th): In- 
direct or disguised reference to the inadequacy of 
therapy or therapist. This includes indirect or subtle 
minimizations of the benefits being derived from 
therapy, Thes atements аге not made with any refer- 
ence to the client's. own feelings or thoughts about 
therapy or therapist, or the client is not taking any 
definite stand or position in his statement. 

Statements indicating unwillingness to 
therapeutic nits or demands when this unw 
is only indirectly expressed, Any reason—other than 
open adr on of not wanting to do so—for not ар- 
pearing on time. Indecision about attending the next 
interview without stating any desire not to, 

8. Inability to understand (U): Client statements 
indicating feigned or real inability to understand the 
therapist When the therapist's statement is judged to 
he clear in meaning. Requests for clarification of such 
therapist statements are included here, but not requests 
for further elaboration (which are included in De- 
pendence). 

Note: A rejection of a therapist's statement ог in- 
terpretation is not considered resistance, unless it also 
happens to meet one of the above criteria. 

The following statements are some examples to clarify 
the categorization of Covert Resistance: 

"I can't define how I felt in that situation." 
“BI” (Bloc! ing). 

2. Therapist: "I wonder where they get that idea?” 
Client: "I don't know." , . . N. Client's inabi ty to 
provide labe motivations, etc, for other people is not 
considered resistance. 

3. "If I could be hypnotized that would help." , 
CR. This is an indirect reference to inadequacy of 
therapy. 

4. “And I was going to tell him off, so I wi 
there about two o'clock (pause), or was it 
(pause), T E34 pues o'clock, and I said to him," 

1 erbalization, grouping for ir- 
relevant detail, 


5. “Since I talked about this thi vi 1 

much better in class than 1 had p ELT Ur 
MAE res uu E мав ting somewhere else this 
doi us “тв”у, ңы E (Ж 3. 
of the benefits being devo ee EER 
‚ 6. Therapis “You felt uncomfortable 
tion.” Client: “What did you say?” 
real or feigned inabili 
statement 


7. "I don't know if I'] 
I have an exam coming up.” 
Rives reason, other than 


dyes. h not wanting to, 
sibility of not coming to therapy session. E А 


ent down 
three o'clock 


in that situa- 
<. CR, This is 
ly to understand a clear therapist 


for pos- 


OPEN RESISTANCE 
“open resistance” 
rbalizes criticism—j 
the therapeutic tech 


Briefly defined, 
which the client ve 


is the extent to 
of the therapist or 


n an open way— 
hnique. Also, per. 
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i iti taying within the 
verbalized opposition to si h 
йш E eae ee eon et аш. therapy which the 
ee P Mp pott unwillingness as 
clien А 


the conditions and limits of the kind of therapy the j 
hi c "AN examples to clarify 
i ategorization of Open Resistance: gue 
Mq uM M QE. xl UN same things eps how this is going to help very much 
pede htianed in pe resistance," but here these — Op. Res. х far is I've been 
things ire admitted d y the client. “АП that's been happening so far poa с: 
anga A pe H Hen attitudes about the therapist — ansy ering questions, when am I going to get 
Е TH dota] Statements are frequently. Swers Op. Res. T€ 
ES E база don of doubts, sarcastic remarks, and "Isn't there some way we could speed this 
only Eny veiled criticism of the therapy or therapist, ... Op. Res, 
These statements admittedly convey 


the thoughts ог 


i right 
4. That’s something I'd rather not talk about 
3 i imsel ient is taki j^. „Өр, Res. tile. 
e f the client himself and the client is taking now ma 7 awhile 
TR ош his opinions or doubts, 5. "I'd like to take a break from uis for a eni it 
NU айы unwillingness (not “inability”) рш you again when I want another apt 

"talk, or discuss any particular area, or to follow that's OK. with Xe" „ „р; Bes 
to talk, Š " 

'THE TEST OF CLINICAL, JUDGMENT 
shed for / 
DIRECTIO him and what isn't. He needs to be pu 
This is а test of your clinical judgment, It con- 


his own good, 
sists of statements made by all sorts of Person: 


couldn't 
S. Your 5. Don't try to tell me that most everybody MA it. j 
task is to read each statement carefully and then make cheat on. tests if they could E 
a judgment about the emotional adjustment of the They’ Te just too scared that's all. аара йет af 
person who made the statement, Do not evaluate the 6. Rich people are Renerally sex ан bearing ® 
content of the statement itself, : 7. We Women are useless for anything ex 
If you can conceive of a statement as coming from „children and keeping house, ins for your soul 
an essentially normal person, place a check under the 8. Good and evil are constantly battling 3 
column marked “A.” If you Cannot conceive of the state. аз well as mine, (ho get 50 
ment as coming from an essentially Normal person, place 9. Old maids are frustrated old women s t of per- 
a check under the column marked “B.” Place one check eager {ог sex tha they practice a lot 
in front of each statement, versions, ү 
» . to. Have you ever heard of them putting a Morg they 
“NORMAL” PIQUE, ттын Rockefeller in jail? You're damned. right 
T. Money can become an important thing in anyone's don't, Only the little Hiss sa. did 
life when he's really Short of it, 11. Survival of the fittest is the ime: of God. 
2. Farmers have as much common sense as other Wrong for us to try to keep weaklings ave ies | 
people. р 12. Ошу а woman сап ‘really understand sO ен. { 
3. The threat of pneumonia is much less marked today feel about love, Men are such insensitive i say ! 
than it once bia 7 P 13. Tf a man can't hold down an honest job, then 
4. A person doesn’t like to think that his home town let him starve, D бё A 
has turned out to be a dump, 14. T advocate celiba feel that there's little T 
5. One of the roughest things about the military life 
is that you have to do so man; 


Or sex 


1 
m professional life, 
¥ apparently senseless 15 i 
things, 


-— HOS 
il that makes People do had mms 
i А e had people. | 
6. It's hard for most of us to realize what it is like 16. но із аА ai a ше, must pe 
to go for days without getting enough food, Whores at heave Кр пува a ildren 
ABNORMAL” FILLER ITEMS 12: All of this nonsense about not frustrating. at have 
1. Little by tittle, drop by drop, he doctors drain the MENT 0 RO ROO. God meant Tor Ue ould 
blood from you "til you are completely bloodless, have set ЫС don't want to do oF i “ag 
2. The men in Moscow have mind machines which they 18, No ri ch s shings шеше way he “did, jeppin û 
use to control anybody they want to control, over тап could Ket rich without $ i 
3. Everybody should really shower ro ж. ЭШ over a lot of People on the way. — D 
day to be absolutely sure that his body is free of 9. T know а person js dishonest, even o! р | | 
killer germs, y TN never trust him again, ш су desks 
4. It's because they've. been so sinful that mom “© ТЕ they make more money at their fan did you 
bodies start to rot inside and you can smell the D eum you do gut here in the yard, but за white 
5. They should send all of the icehoxes and тегин. all look close at them? Theyre oft ane bal | 
ators to Alaska and everyone should Ko up ere all Over—just like women, They're no! 1 
and live in one with his mother and father, С 21. Ge bunch of fruits, you ene 
6. Everyone is doomed, Very soon now, the earth wi 277 Getting the JUMP on the next guy so E Р 
open up and we'll all pe swallowed into Hell a a n is the main thing about living M 
" ‘orld, the wa: it is, А F 
TEST ITEMS 22. 105 dangerous 6 young boys and girls, ge is 
1. My life has been laid out for mg and Tc. | gether hy themselves, The first thing they 
mo more than live it as hay pet Planned." ° аа Playing around with enh sehen. forgive 
a. Wives аш be traded ua prem fairly fr “People who get ‘sick eho pray to God to fof of 
„intervals. Otherwise things дер boring, duent them for their sins, Sickness is God's W? 
3. You can usually tell whet; а kid is going to 24, To ishing us for our wicked ways. f think 
out to be a bad egg or not and if he is You mien 74 3 gotta be a Pretty queer guy in my way 0 
as well lock him up right then re want E who would ever knit anything. ‚ stuff 
he nearly so many killings nd © wouldn't 25. They just fill kids’ wn: 1 e lot of fancy * oq 
lock them up and Bet rid of the ungs if weg ае т S with d aaa aret 
1. The ordinary Person doesn m, eges so that their old is 
x de gre Swt know What's good for 6 стой for them anymore, handle him ee 
E = Бе! a wild kid y мау па k 
"Grouped for the reader's Convenience to break а ае оу: ay: to s: 


3 » cri 
i т. Otherwise he'll end up а 
Something. 


EFFECTS ОЕ A REFLECTIVE 


27. Listen, take my advice. Don't be fooled by the soft, 
U delicate appearance women make. They'd take 
over this world and make slaves of us men if 
we'd relax our control just one little bit. 

They ought to open the floodgates and flood the 

whole damned South with everyone in it. 

29. Don't talk to me about those clergymen, with their 
“holier than thou" attitudes. They'd lie, cheat, 
and steal just as much as anybody. А 

30. Parents always know what's right, so it's best not 
to do anything without first asking them. 

31. Most college graduates are radicals, homosexuals, 
or troublemakers. 

32. Girls who go around screwing with the boys ought 
to be whipped in public and made an example of. 

33. Those people in the slums are basically weak in 
character and you'll never ma them strong like 
good honest workers by giving them more parks 
and fresh air. 

34. Every Ыш corporation and most of the smaller ones 
have company spies mixed in with the workers, 

35. You've really got to make a criminal suffer severely 
for his crime. Punish him until he has no spirit 
left. Then he'll be willing to do what right. 

36. Dancing and card playing are just as sinful sex. 
Better а person should keep himself locked in his 
room than to indulge in these evil 

37. My mother alway d to act like a lady at all 
times, She is right. I've found that if a woman 
lets her guard down for а minute that a “gentle. 
man" will change to an animal, just like that. 

38. Man is basically evil and at the most can do no 
more than to control the evil that is within him. 

зо. It's bad business letting women take over men's 
jobs, Why do you think that they hate to have 
them in mines and on ships? Do you think 
just superstition ? 

40. A child must learn who his master is, Т 
of time for him to think and act 
when he's ап adult 

41. If you don't stop those people from talking with 
each other, you will soon find that they have 

formed some sort of conspiracy which you will 
have a hard time suppressing. 

We should destroy criminals because they'll 


it's 


here's plenty 
for himself 


just 


POSTTHERAPY 
DIRECTION 

In rating the client on each item of the scale, 
sider the difference between the w. 
he entered therapy and the way he was at the end of 
treatment. Base your rating on the change that has oc- 
curred in the client within this interval, 

On certain items you may feel that a client came 
into therapy with a behavior already incorporated as 
part of his usual behavior and that he has not changed 
in this particular area. On these items, clients will be 
rated “remained the same,” For example, a client 
may have been quite willing to discuss the nificant 
aspects of his problems from the time he entered 
therapy, and he has made no ch: nge in this respect. 

The scale consists of two kinds of items. Most of 
the items deal with behavior which is not specifically 
related to the counseling situation; however, certain 
items are based on the client's behavior in therapy. 

On those items which are related to over-all or ven. 
eral behavior, your rating should reflect changes in the 
behavior and/or characteristics of the client which you 
feel are relatively permanent and which also apply to 
his behavior in situations D. On diese 
items indicate onlv changes which vou feel the client 


1 con- 
the client was when 


AND A LEADING 


3 


jd: 
I 


ši 


. The big shots 
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produce more of their own kind. 

I tell you it bad to send 18-year-olds to the 
Army. They'll turn them out as killers, cynics, 
homosexuals—or worse. 

Laborers are a bunch of grea dirty slobs. 

If it was a matter of sacrificing some of my hap- 
piness to help out someone else, I wouldn't do it. 

Who would criticize the President of the United 
States except Communists, homosexuals, extrem- 

ts, and that sort. 

Who do these women think the kidding with 
all of their modesty and morals? They? jump into 
bed with the first guy that asked them if they 
thought they could get away with i 

Killing dogs and monkeys is a horrible thing—no 
matter how much good they claim will come of it. 

in get away with anything. But just 

don't let the little man step over the line, 

Bachelors are like vicious animals w 
on any young girl they can and 
of her. 

І don't see why we should tax the people who are 
smart enouzh to make out in order to take care 
of those who are too dumb or too damned lazy 
to take care of themselves, 

When somebody does not enforce 
one more bit of disorganiz 
things like that which cre; 
for everybody, 

You gotta be a bigger crook than they are; 
the only way 
shots, 

There is no such thing asa‘ 


iting to pounce 
uke advantage 


a rule, it adds 
ton to things. It's 
te no end of trouble 


that 
you'll get the respect of those big 


"white lie," Any devia 

tion from the exact truth, no matter how ins g 
nificant it may seem, fosters evil. 

Don't let 'em fool you with these “J 
you" spiels. They got an angle 
want to use уоп, 

A woman who would work as a bartender must he 
nothing but a prostitute. What other reason would 
she have for taking a job like that? 

A company union, ha! You know what that's for 
don't vou? So that the bos an run it and bleed 
you for all you're worth. 


want to help 
and they just 


RATING SCALE 


client's 
within 
mere conformity. 
In certain cases, 
client's behavior in 
ог appropriate. For example, 


n his evaluation of other 
tton 


in 


а more critical 
client had been һу 


ров 


client may have 


ie 


8: 


attitudes, 


feelings, and 
therag 


Your rating 


motivations 


operating 
should not 


be based on 


you are asked to rate ch 


anzes in the 
terms of whethe 


| er they are realist 
if a client was too positive 
people, a more realistic posi- 
from the client's developing 
е toward other People. Tf the 
Percritical of others, a more realistic 


be based on th 
ах we extent to which th 
become less critical of others x 


might have resulted 
attitude 


Чоп would 


The client's perception of 
of his own behavior ha. 
(а) become less realistic; (5 
(c) become slightly more 
moderately more realistic: 
ably more realistic 0S 

The client's placing 
culties on others 
stances has: 


as а function 


) remained the same; 
realistic; (d) become 
(e) become consider. 


of responsibil 
and/or eny 


ity for his diff- 
ironmenta] 


circum- 
(a) become considerably 
eut moderately тоге D А 
Slightly more realisti a 
y alisti (d) т 

ae become less realistic, males на ip 

Кы A 3 
€ extent to which the client Derceives th 1 
* the prob. 


m merely in symptomatic terms has: 
n Ён: 861 considerably ; (b) decreased mod- 
lerately; (c) decreased slightly; (d) remained. 
the same; (с) increased. ae 
4. The client’s feelings of discomfort in his everyday 
life have: (a). decreased considerably; (b) de- 
creased moderately; (c) decréased slightly; (d) 
remained the same; (e) increased. d 
5. The client’s expression of positive emotions when 
they are appropriate has: М 
(a) decreased; (b) remained the same; (c) in- 
creased slightly; (d) increased moderately; (c) 
increased considerably. 
6. The client's symptoms have: * : 
` (a) become considerably less disturbing; (b) be- 
come moderately less disturbing; (e) become 
slightly less disturbing; (d) remained the same; 
(с) become more disturbing. ў 
7. The client discusses feelings and attitudes which 
are relevant to the problem: 
(a) considerably more freely; (b) moderately 
more freely; (c) slightly more freely; (d) the 
same; (e) less freely, 
8. The client's comprehension of important causal 
relationships between his symptoms and the 
' underlying needs and conflicts has: 
2 (a) decreased; (b) remained the same; (c) in- 
creased slightl ; (d) increased moderately; (e) 
increased considerably. 
9. ient's avoidance of making decisions 
Scem to be necessar: 
4 ccomplished has: 


(а). decreased considerably; (b) detreased mod- 


erately; (c) decreased slightly; (d) remained the 
same; (е) increased, 

10. The client's attitudes toward other people have: 
(a) became less Appropriate; (b) remained the 


same; (c) become slightly more appropriate; (d) 
become moderately more appro 


n priate; (с) become 
considerably more appropriate, — 
11. Concerning his ability to solve his problems, the 
-client has: d 
(a) become less confident; 
confident; (c) become slightly more confident; 
` (d) become moderately more confident; (e) be- 
come considerably more confident. 
12. The client's understanding of his problems in terms 
= 6f his own past experience has: 
(а) become considerably 
become moderately тоге 
come slightl. 
same; (e) 
13. The client fe 
_ (a) 


which 
у before anything can 


(b) become no more 


more meaningful; (b) 


hips with others have: 
(b) remained the 
more satisfactory; 


б € satisfactory; (e) 
considerably more satisfactory, à 


develops new plans 


d considerably; 
erately; (c) increased slightly 
- Same; (е) decreased, 

hs. c feelings and attitudes toward himself 


(а) become considerab]: i 
lc ly more appropriate; (p 
become moderately more appropriate; (c) Же 
. slightly more appropriate; (d) remained the same; 
0 (еу become less appropriate, j 
16. The client's expression of negative emotions when 
4 


(b) increased mod. 
; (d) remained the 
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they are appropriate has: (a) decreased; (5) T 
mained the same; (c) increased ightly e 
increased moderately; (e) increased considerably. 
17. The client's view of his strengths and shortcom 
ings has: i : Р 
(a) become less realistic; (b) геп ined сне same 
(c) become slightly more realistic; (d) eco Н 
moderately more realistic; (e) become conside! 
ably more realistic. 
18. The client's emotional 
and situations have: u^ Үү 
(a) become considerably more realistic; ( ) К 
come moderately тоге realistic; (c) ер 
slightly more realistic; (d) remained the same; 
(е) become less realistic. = 
i i i ms 
19. The client's understanding of his problems in ter 
of his own needs has: фу. Не 
(a) become considerably more accurate; eris 
come moderately more accurate; (с) pm 
slightly more accurate; (d) remained the sé 
(e) become less accurate. " 
ient' i i i nter- 
20. The client's expectations for himself in the i 
personal aspects of his life have: я ilis 
(a) become less realistic; (b) малан, as 
same; (c) become slightly more Sens E rg 
come moderately more realistic; (e) beco! 
siderably more realistic. " wing 
sibility solving 
. The client's acceptance of responsibility for 
his problems has: ji 
А, S inc 5 тос 
(а) increased considerably; (P) Г ч ШЕ 
erately; (c) increased slightly; (d) rema 
same; (с) decreased. ЖЕТ, 
The client's perception of the meaning of the 
tions of others toward him has: 1; (b) be 
(а) become considerably less distorted: рш 
come moderately less distorted; ut ает 
slightly less distorted; (4) remained the 5а 
(с) become ee pea " 
i 8 involvement as 
23. The client's meaningful emotional ОНА сар 
opposed to an intellectual approach to his 


reactions to other people 


ы 
N 


lems has: dy 
(а) increased considerably; (b) increased тос 
erately; (c) increased slightly; (d) remained the 


same; (е) decreased. 
24. The client's attempts to avoid significant areas of 
discussion has: c) de- 
(a) increased; (b) remained the same; M 85 
creased slightly; (d) decreased moderately ; 
decreased considerably, is own 
25. The extent to which the client sees. where ШИА e 
characteristic ways of thinking, feeling, ап his 
having bear an important relationship to 
problems has: ‚ (д in 
(a) decreased; (b) remained the MEUM 2: (e) 
creased slightly; (d) increased moderately ; 
increased considerably. ng as re 
. The client's understanding of his probleme, other 
sulting from interpersonal relationships W 
people has: ined the same; 
(a) become less realistic; (b) тепле, become 
(c) become slightly more realistic; e consider- 
moderately more realistic; (c) becom 
ably more realistic. andling 
27. The client's attempts to try new ways of handlin 
his problems have: x (0), 
(a) decreased; (b) remained the т Ау (е) 
creased slightly; (4) increased moderately; 
increased considerably, 


in- 


* 
* dier x] 
і s" pe 
TIME od 
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